You are on page 1of 90

Fundamentals of Testing

Methods and Techniques of Automotive Software


Testing

2.11.2022. Doc. dr. sc. Ivan Vidović 1


Contents
• Why Software Fail?
• Why is Testing Necessary?
• What is Testing?
• Seven Testing Principles
• Fundamental Test Process
• Test planning and control
• Test analysis and design
• Test implementation and execution
• Evaluating exit criteria and reporting
• Test closure activities
• Test Documentation
• Severity & Priority in Testing
• The Psychology of Testing
• Code of Ethics

2.11.2022. Doc. dr. sc. Ivan Vidović 2


Why software fail?

2.11.2022. Doc. dr. sc. Ivan Vidović 3


What is a „bug”
• Error: a human action that produces an incorrect
result
• Fault: a manifestation of an error in software
• also known as a defect or bug
• if executed, a fault may cause a failure
• Failure: deviation of the software from its expected
delivery or service
• found defect
• Failure is an event; fault is a state of the software,
caused by an error
2.11.2022. Doc. dr. sc. Ivan Vidović 4
Error-Fault-Failure

2.11.2022. Doc. dr. sc. Ivan Vidović 5


Reliability versus faults
• Reliability: the probability that software will not
cause failure of the system for a specified time
under specified conditions

• Can a system be fault-free? (zero faults, right first time)


• Can a software system be reliable but still have faults?
• Is a “fault-free” software application always reliable?

2.11.2022. Doc. dr. sc. Ivan Vidović 6


Why do faults occur in software?
• Software is written by human beings
• who know something, but not everything
• who have skills, but aren’t perfect
• who do make mistakes (errors)
• Under increasing pressure to deliver to strict
deadlines
• no time to check but assumptions may be wrong
• systems may be incomplete
• If you have ever written software…

2.11.2022. Doc. dr. sc. Ivan Vidović 7


What are the consequences of
software faults?
• Huge sums
• Ariane 5 ($7 bilion)
• Mariner space probe to Venus ($250 m)
• American Airlines ($50 m)
• Very little or nothing at all
• minor inconvenience
• no visible or physical detrimental impact
• Software is not „linear”
• small input may have very large effect

2.11.2022. Doc. dr. sc. Ivan Vidović 8


Most common software problems
• Incorrect calculation
• Usually in date and financial calculations
• Incorrect data edits
• The day of the month limited to 31
• Ineffective data edits
• Leading spaces allowed as input
• Incorrect coding/implementation of business rules
• Incorrect, missing, or unclear requirements specifications
• Misinterpretation of requirement specification
• Inadequate software performance
• Slow system response
• Confusing or misleading data
• Presented data may be correct, but the users do not understand it
• Software that is difficult to use
• Difficult to navigate in the software
• Several steps required to perform simple task
• Could be result of not planned and coordinated modifications

2.11.2022. Doc. dr. sc. Ivan Vidović 9


Most common software problems
• Obsolete software
• Software is based on functions from older version of operating system
• Based on older hardware
• Inconsistent processing
• Software works only in one environment
• Sometimes, this can be feature, but usually software should be portable and used in different
environments
• Difficult to maintain and understand
• Programmer needs to analyze and understand the software before he can upgrade and maintain it
• Problem with unstructured code/programs
• Unreliable results or performance
• Results are not consistent
• Inadequate support of business needs
• It is not flexible enough to satisfy busines needs
• Lack of features to customize business rules
• No longer support by the vendor
• Product is not supported by vendor, vendor is out of business or product is sold

2.11.2022. Doc. dr. sc. Ivan Vidović 10


Most common software problems
• Incorrect or inadequate interfaces with other systems
• Does not correctly receive input from other systems or does not correctly send output to other systems
• Incorrect matching and merging of data
• Matching or merging dana from different sources based on ID number
• Data searches that yields incorrect results
• Results display correct employee name, but timesheet from other employee
• Incorrect processing of data relationship
• Post office code which is inadequate for the address
• Incorrect file and data handling
• Wrong data from right source
• Wrong data from wrong source
• Inadequate security controls
• Unauthorized access is not controlled and detected adequately
• Error in user roles in software
• Inability to handle production data capacities
• Software can process only $9 999 999,99 but company needs to process transaction over $10 million
• Year 2000 problem

2.11.2022. Doc. dr. sc. Ivan Vidović 11


Most common software problems
• It is big challenge to cover all mentioned software
problems
• Possible solutions
• Design a risk questionnaire about each problem
• Ask users to assess the risk impact
• Assign priorities according to the past defect reports

2.11.2022. Doc. dr. sc. Ivan Vidović 12


Need for testing
• Testing ensures that what you get is what you wanted
to build
• Enhances the integrity of a system by detecting
deviations in design and errors in the system

• In short:
1. To check the software reliability
2. To be sure that the software does not contain any
bug which can become a reason for failure
3. The software is created according to its specification.

2.11.2022. Doc. dr. sc. Ivan Vidović 13


Need for testing
• Common things that creates the need for testing
1. Communication gaps between the requirement
specifier and the developer
2. Not proper understanding of all the complexities
of the system
3. Inadequate requirements gathering

2.11.2022. Doc. dr. sc. Ivan Vidović 14


Why is testing
necessary?

2.11.2022. Doc. dr. sc. Ivan Vidović 15


Why is testing necessary?
• Software system context are an increasing part of life, from
business applications to consumer products
• Causes of software defects:
• Defects occur because human beings are fallible
• Failures can be caused by environmental conditions
• Role of testing in software development, maintenance and
operations:
• Rigorous testing of systems and documentation can help to reduce the
risk of problems occurring in an operational environment
• Testing and quality:
• With the help of testing, it is possible to measure the quality of
software in terms of defects found
• How much testing is enough?
• Deciding how much testing is enough should take account of the level
of risk

2.11.2022. Doc. dr. sc. Ivan Vidović 16


Testing and quality
• Testing help us to measure the quality of software
in terms of number of defects found, the tests run,
and the system covered by the tests.
• To deliver quality product developers need correct
specifications.
• The aim of the projects is to deliver software
according to the specifications.
• Project also need to satisfy budget and timescale
requirements

2.11.2022. Doc. dr. sc. Ivan Vidović 17


Validation vs Verification
Validation Verification
• Testing and validating if the • Checking if the software has
software product meets the been built according to the
needs of the customer or not requirements or not
• Ensure that the software fulfills • The main goal is to ensure quality
the desired use in an appropriate of software application, design,
environment architecture
• Activities like unit testing, • Activities like reviews, walk-
integration testing, system throughs and inspection.
testing and user acceptance • ISTQB definition: Confirmation
testing by examination and through
• ISTQB definition: Confirmation provision of objective evidence
by examination and through that specified requirements have
provision of objective evidence been fulfilled
that the requirements for a
specific intended use or
application have been fulfilled.

2.11.2022. Doc. dr. sc. Ivan Vidović 18


Why is testing necessary?
• Because software is likely to have faults
• To learn about the reliability of the software
• To fill the time between delivery of the software
and the release date
• To prove that the software has no faults
• Because testing is included in the project plan
• Because failures can be very expensive
• To avoid being sued by customers
• To stay in business

2.11.2022. Doc. dr. sc. Ivan Vidović 19


What is testing?

2.11.2022. Doc. dr. sc. Ivan Vidović 20


What is software testing?
• Software testing can be stated as the process of
validating and verifying that a software
program/application/product:
• meets the requirements that guided its design and
development
• works as expected

2.11.2022. Doc. dr. sc. Ivan Vidović 21


What is testing?
• Definition:
• Testing is process of trying to discover every
conceivable fault or weakness in a work product.
• Testing is a process of executing a program with the
intent of finding an error.
• A good test is one that has a high probability of
finding an undiscovered error.

2.11.2022. Doc. dr. sc. Ivan Vidović 22


Debugging
• Definition:
• Debugging is the routine process of locating and removing
computer program bugs, errors or abnormalities, which is
methodically handled by software programmers via
debugging tools. Debugging checks, detects and corrects
errors (or "bugs") to allow proper program operation,
according to set specifications.
• Unknowns during debugging:
• Is the cause of the bug reproduced in other program
module/part?
• Does fix of current bug introduce „new bug”?
• What could have been done to prevent this bug?

2.11.2022. Doc. dr. sc. Ivan Vidović 23


Exhaustive testing
• Also known as complete testing
• Testing technique in which all scenarios are tested
• In real project/scenario it is impossible to test all
combinations of inputs and preconditions
• It takes impractical amount of time

2.11.2022. Doc. dr. sc. Ivan Vidović 24


How much testing is enough?
• It’s never enough
• When you have done what you planned
• When your customer/user is happy
• When you have proved that system works correctly
• When you are confident that the system works
correctly
• It depends on the risks for your system

2.11.2022. Doc. dr. sc. Ivan Vidović 25


Testing and quality
• Testing measures software quality
• Testing can find faults; when they are removed,
software quality (and possibly reliability) is
improved
• What is tested during testing
• system function, correctness of operation
• non-functional qualities: reliability, usability,
maintainability, reusability, testability, etc.

2.11.2022. Doc. dr. sc. Ivan Vidović 26


Other factors that influence
• Contractual requirements
• Legal requirements
• Industry-specific requirements
• e.g. pharmaceutical industry (FDA), compiler standard
tests, safety-critical or safety-related such as railroad
switching, air traffic control

2.11.2022. Doc. dr. sc. Ivan Vidović 27


Software testing limits
• Due to testing time limit, it is impossible to achieve total
confidence.
• We can only test against system requirements.
• We can never be sure the specifications are 100% correct.
• We can never be certain that testing system (or tool) is correct.
• There is no testing tools which can test every software program.
• Test engineers can never be sure that they completely understand
a software product.
• We never have enough resources to perform software testing.
• Compromise between thoroughness and budget.
• Even if you do find the last bug, you will never know it.
• We can never be certain that we achieve 100% adequate
software testing.

2.11.2022. Doc. dr. sc. Ivan Vidović 28


Who tests the software?

Developer Independent tester


• understands the system • must learn about the
• will test „gently” system
• is driven by „delivery” • will attempt to break it
• is driven by quality

2.11.2022. Doc. dr. sc. Ivan Vidović 29


Software testing terminologies
• Accuracy: The capability of the software product to
provide the right or agreed results or effects with the
needed degree of precision.
• Actual result: The behavior produced/observed when a
component or system is tested.
• Behavior: The response of a component or system to a
set of input values and preconditions.
• Component: A minimal software item that can be
tested in isolation.
• Debugging: The process of finding, analyzing and
removing the causes of failures in software.

2.11.2022. Doc. dr. sc. Ivan Vidović 30


Software testing terminologies
• Deliverable: Any (work) product that must be delivered
to someone other than the (work) product’s author.
• Error: A human action that produces an incorrect
result.
• Expected result: The behavior predicted by the
specification, or another source, of the component or
system under specified conditions.
• Fail: A test is deemed to fail if its actual result does not
match its expected result.
• Failure: Actual deviation of the component or system
from its expected delivery, service or result.

2.11.2022. Doc. dr. sc. Ivan Vidović 31


Software testing terminologies
• Integration: The process of combining components or
systems into larger assemblies.
• Pass: A test is deemed to pass if its actual result matches its
expected result.
• Priority: The level of (business) importance assigned to an
item, e.g. defect.
• Severity: The degree of impact that a defect has on the
operation of a component or system.
• Tester: A technically skilled professional who is involved in
the testing of a component or system.
• Use Case: The specification of tests that are conducted from
the end-user perspective. Use cases tend to focus on
operating software as an end-user would conduct their day-
to-day activities.
2.11.2022. Doc. dr. sc. Ivan Vidović 32
Seven testing principles

2.11.2022. Doc. dr. sc. Ivan Vidović 33


Seven Testing Principles
1. Testing shows presence of defects.
2. Exhaustive testing is impossible.
3. Early testing.
4. Defect clustering.
5. Pesticide paradox.
6. Testing is context dependent.
7. Absence of errors is fallacy.

2.11.2022. Doc. dr. sc. Ivan Vidović 34


Is the software defect free?
Testing Principle – Testing shows presence of defects
Testing can show that defects are present but cannot prove
that there are no defects. Testing reduces the probability of
undiscovered defects remaining in the software but, even if no
defects are found, it is not a proof of correctness.
• We cannot say “All swans are white”.
• However, as soon as we see one black swan we can say “Not
all swans are white”.
• In the same way, however many tests we execute without
finding a bug, we have not shown “There are no bugs”.
• As soon as we find a bug, we have shown “This code is not
bug-free”.

2.11.2022. Doc. dr. sc. Ivan Vidović 35


How much testing is enough?
Testing Principle – Exhaustive testing is impossible
Testing everything (all combinations of inputs and
preconditions) is not feasible except for trivial cases.
Instead of exhaustive testing, we use risks and priorities
to focus testing efforts.
• Assessing and managing risk is one of the most
important activities in any project and is a key activity
and reason for testing.
• Deciding how much testing is enough should take
account of the level of risk, including technical and
business risks related to the product and project
constraints such as time and budget.
2.11.2022. Doc. dr. sc. Ivan Vidović 36
When can we meet our test
objectives?
Testing Principle – Early testing
Testing activities should start as early as possible in the
software or system development life cycle and should be
focused on defined objectives.
• We can use both dynamic testing and static testing as a
means for achieving similar test objectives.
• Testing can have different goals and objectives, which
often include:
• Finding defects
• Gaining confidence in and providing information about the
level of quality
• Preventing defects.

2.11.2022. Doc. dr. sc. Ivan Vidović 37


Focusing on defects can help us
plan our tests?
Testing Principle – Defect clustering
A small number of modules contain most of the
defects discovered during pre-release testing or show
the most operational failures.
• Reviewing defects and failures in order to improve
processes allows us to improve our testing and our
requirements, design and development processes.
• One phenomenon that many testers have observed
is that defects tend to cluster.

2.11.2022. Doc. dr. sc. Ivan Vidović 38


The defect clusters change over
time
Testing Principle – Pesticide paradox
If the same tests are repeated over and over again, eventually
the same set of test cases will no longer find any new bugs. To
overcome this “pesticide paradox”, the test cases need to be
regularly reviewed and revised, and new and different tests
need to be written to exercise different parts of the software
or system to potentially find more defects.
• Over time, as we improve our whole software development
life cycle and the early static testing, we may find that
dynamic test levels find fewer defects.
• A typical test improvement initiative will initially find more
defects as the testing improves and then, as defect
prevention kicks in, we see the defect numbers dropping.
2.11.2022. Doc. dr. sc. Ivan Vidović 39
Software system context
Testing Principle – Testing is context dependent
Testing is done differently in different contexts. For
example, safety-critical software is tested differently
from an e-commerce site.
• Software systems are an increasing part of life,
from business application (e.g. banking) to
consumer products (e.g. cars).
• Software that does not work correctly can lead to
many problems, including loss of money, time or
business reputation, and could even cause injury or
death.

2.11.2022. Doc. dr. sc. Ivan Vidović 40


Absence of errors
Testing Principle – Absence of errors fallacy
Finding and fixing defects does not help if the system
built is unusable and does not fulfill the users’ needs
and expectations.
• If we don’t find defects does that mean the users
will accept the software?
• We must consider the customers for software.
• The people using software want software which
will meet their needs.

2.11.2022. Doc. dr. sc. Ivan Vidović 41


Fundamental test
process

2.11.2022. Doc. dr. sc. Ivan Vidović 42


Fundamental test process
• The activities within the fundamental test process
fall into the following basic steps:
• Planning and control
• Analysis and design
• Implementation and execution
• Evaluating exit criteria and reporting
• Test closure activities

2.11.2022. Doc. dr. sc. Ivan Vidović 43


Test planning and control
• Sometimes called the mission of testing or the test
assignment.
• Set the objectives for the testing itself, and derive an
approach and plan for the tests, including specification
of test activities.
• To help us we may have organization or program test
policies and a test strategy.
• Test policy gives rules for testing, e.g. “we always
review the design documents”.
• Test strategy is overall high-level approach, e.g. “system
testing is carried out by an independent team reporting
to the program quality manager. It will be risk-based
and proceeds from a product (quality) risk analysis.”

2.11.2022. Doc. dr. sc. Ivan Vidović 44


Test planning
• Test planning has the following major tasks, which help
us build a test plan:
• Determine the scope and risks and identify objectives of
testing
• Determine the test approach (techniques, test items,
coverage, identifying and interfacing with the teams involved
in testing, testware)
• Implement the test policy and/or the test strategy
• Determine the required test resources (e.g. people, test
environment, PCs)
• Schedule test analysis and design tasks, test implementation,
execution and evaluation
• Determine the exit criteria

2.11.2022. Doc. dr. sc. Ivan Vidović 45


Test control
• Test planning considers the feedback from monitoring
and control activities which take place during the
project.
• It includes comparing actual progress against the plan,
and reporting the status, including deviations from the
plan
• Test control has the following major tasks:
• Measure and analyze the results of reviews and testing
• Monitor and document progress, test coverage and exit
criteria
• Provide information on testing
• Initiate corrective actions
• Make decisions

2.11.2022. Doc. dr. sc. Ivan Vidović 46


Test analysis and design
• Test analysis and design is the activity where general testing
objectives are transformed into tangible test conditions and
test designs.
• Test analysis and design has the following major tasks in the
following order:
• Review the test basis. The test basis is the information on which
test cases are based, such as requirements, design specifications,
product risk analysis, architecture and interfaces.
• Identify test conditions (based on analysis of test items, their
specifications, and what we know about their behavior and
structure)
• Design the tests.
• Evaluate testability of the requirements and system.
• Design the test environment set-up and identify any required
infrastructure and tools.

2.11.2022. Doc. dr. sc. Ivan Vidović 47


Test implementation and
execution
• During test implementation and execution, we take
the test conditions and make them into test cases
and testware and set up the test environment.
• We also need to set up an environment where we
will run the tests and build our test data.

2.11.2022. Doc. dr. sc. Ivan Vidović 48


Implementation
• Major tasks:
• Develop and test cases by using techniques and create
test data for those tests
• Create test suites from the test cases for efficient test
execution. Test suite is a collection of test cases that are
used to test a software program
• Implement and verify the environment.

2.11.2022. Doc. dr. sc. Ivan Vidović 49


Execution
• Execute the test suites and individual test cases, following our
test procedures. We might do this manually or by using test
execution tools.
• Log the outcome of test execution and record the identities and
versions of the software under test, test tools and testware.
• Compare actual results (what happened when we run the tests)
with expected results (what we anticipated would happen).
• Reporting defects. Where there are differences between actual
and expected results, report discrepancies as incidents
• Repeat test activities as a result of action taken for each
discrepancy. It covers:
• Confirmation testing or re-testing: We need to re-execute tests that
previously failed in order to conform a fix.
• Regression testing: “Testing in order to verify that the programmers did
not introduce defects in unchanged areas of the software and that
fixing a defect did not uncover other defects.

2.11.2022. Doc. dr. sc. Ivan Vidović 50


Evaluating exit criteria and
reporting
• Evaluating exit criteria is a process defining when to
stop testing.
• Exit criteria is considered when:
• Maximum test cases are executed with certain pass
percentage
• Bug rate falls below certain level
• When we achieve the deadlines
• Evaluating exit criteria has the following major tasks:
• Check test logs against the exit criteria specified in test
planning
• Assess if more tests are needed or if the exit criteria specified
should be changed
• Write a test summary report for stakeholders.

2.11.2022. Doc. dr. sc. Ivan Vidović 51


Test closure activities
• During test closure activities, we collect data from
completed test activities to consolidate experience,
including checking and filling testware, and
analyzing facts and numbers.
• We may need to do this when software is delivered.
• We also might close testing for other reasons, such
as when
• We have gathered the information needed from testing,
• When the project is cancelled,
• When a particular milestone is achieved,
• Or when a maintenance or update is done.

2.11.2022. Doc. dr. sc. Ivan Vidović 52


Test closure activities
• Test closure activities include the following major tasks:
• Check which planned deliverables we actually delivered.
• Ensure all incident reports have been resolved through defect
repair.
• Document the acceptance of the system.
• Finalize and archive testware, such as scripts, the test
environment, and any other test infrastructure, for later
reuse.
• Hand over testware to the maintenance organization who will
support the software and make any bug fixes or maintenance
changes, for use in confirmation testing and regression
testing.
• Analyze the lessons learnt.

2.11.2022. Doc. dr. sc. Ivan Vidović 53


Misunderstanding about testing
• Testing is debugging
• Testing is not the job of programmer
• If programmers were more careful testing would be
unnecessary
• Testing never ends
• Testing activities start only after the coding is
complete
• Testing is not creative task

2.11.2022. Doc. dr. sc. Ivan Vidović 54


Test Documentation

2.11.2022. Doc. dr. sc. Ivan Vidović 55


What is Test documentation?
• Test documentation is documentation of artifacts
created before or during the testing of software.
• It helps the testing team to estimate testing effort
needed, test coverage, resource tracking, execution
progress…
• It is a complete suite of documents that allows you
to describe and document test planning, test
design, test execution and test results that are
drawn from the testing activity.

2.11.2022. Doc. dr. sc. Ivan Vidović 56


Why test formality?
• It's easy to assume that testing is executing the
various section of code on an ad-hoc basis and
verifying the results
• In the real world, testing is a very formal activity
and is documented in detail.
• Test documentation makes planning, review, and
execution of testing easy as well as verifiable.
• The degree of test formality depends on:
• The type of application under test
• Standards followed by your organization
• The maturity of the development process.

2.11.2022. Doc. dr. sc. Ivan Vidović 57


Why test formality?
• Testing activities generally consume 30% to 50% of
software development project effort.
• Documentations help to identify Test process
improvement that can be applied to future
projects.

2.11.2022. Doc. dr. sc. Ivan Vidović 58


Examples of test documentation
• Test policy
• It is a high-level document which describes principles, methods and
all the important testing goals of the organization.
• Test strategy
• A high-level document which identifies the Test Levels (types) to be
executed for the project.
• Test plan
• A test plan is a complete planning document which contains the
scope, approach, resources, schedule, etc. of testing activities.
• Requirements Traceability Matrix
• This is a document which connects the requirements to the test
cases.
• Test scenario
• Test scenario is an item or event of a software system which could
be verified by one or more Test cases.

2.11.2022. Doc. dr. sc. Ivan Vidović 59


Examples of test documentation
• Test case
• It is a group of input values, execution preconditions,
expected execution postconditions and results. It is
developed for a Test Scenario.
• Test data
• Test Data is a data which exists before a test is executed. It is
used to execute the test case.
• Defect report
• Defect report is a documented report of any flaw in a
Software System which fails to perform its expected function.
• Test summary report
• Test summary report is a high-level document which
summarizes testing activities conducted as well as the test
result.

2.11.2022. Doc. dr. sc. Ivan Vidović 60


Best practice to achieve Test
documentation
• QA team needs to be involved in the initial phase of the project
so that Test Documentation is created in parallel
• Don’t just create and leave the document, but update whenever
required
• Use version control to manage and track your documents
• Try to document what is needed for you to understand your work
and what you will need to produce to your stakeholders
• You should use a standard template for documentation like excel
sheet or doc file
• Store all your project related documents at a single location. It
should be accessible to every team member for reference as well
as to update when needed
• Not providing enough detail is also a common mistake while
creating a test document

2.11.2022. Doc. dr. sc. Ivan Vidović 61


Advantages of Test
documentation
• The main reason behind creating test documentation is to either reduce
or remove any uncertainties about the testing activities. Helps you to
remove ambiguity which often arises when it comes to the allocation of
tasks
• Documentation not only offers a systematic approach to software
testing, but it also acts as training material to freshers in the software
testing process
• It is also a good marketing & sales strategy to showcase Test
documentation to exhibit a mature testing process
• Test documentation helps you to offer a quality product to the client
within specific time limits
• In Software Engineering, Test documentation also helps to configure or
set-up the program through the configuration document and operator
manuals
• Test documentation helps you to improve transparency with the client

2.11.2022. Doc. dr. sc. Ivan Vidović 62


Disadvantages of Test
documentation
• The cost of the documentation may surpass its
value as it is very time-consuming
• Many times, it is written by people who can't write
well or who don't know the material
• Keeping track of changes requested by the client
and updating corresponding documents is tiring
• Poor documentation directly reflects the quality of
the product as a misunderstanding between the
client and the organization can occur

2.11.2022. Doc. dr. sc. Ivan Vidović 63


Severity & priority in
testing

2.11.2022. Doc. dr. sc. Ivan Vidović 64


Bug Severity
• Bug or defect severity is a degree of impact a bug
or defect has on the software under test.
• A higher effect of bug/defect on system
functionality will lead to a higher severity level.
• A Quality Assurance engineer usually determines
the severity level of a bug/defect.

2.11.2022. Doc. dr. sc. Ivan Vidović 65


What is priority?
• Priority is defined as the order in which a defect
should be fixed. Higher the priority the sooner the
defect should be resolved.
• Defects that leave the software system unusable
are given higher priority over defects that cause a
small functionality of the software to fail.

2.11.2022. Doc. dr. sc. Ivan Vidović 66


Types of severity
• Critical
• This defect indicates complete shut-down of the
process, nothing can proceed further
• Major
• It is a highly severe defect and collapses the system.
However, certain parts of the system remain functional
• Medium
• It causes some undesirable behavior, but the system is
still functional
• Low
• It won't cause any major break-down of the system

2.11.2022. Doc. dr. sc. Ivan Vidović 67


Types of priority
• High
• The defect must be resolved as soon as possible as it
affects the system severely and cannot be used until it is
fixed
• Medium
• During the normal course of the development activities
defect should be resolved. It can wait until a new version
is created
• Low
• The Defect is an irritant, but repair can be done once the
more serious defect has been fixed
2.11.2022. Doc. dr. sc. Ivan Vidović 68
Tips for determining the Severity
of a defect
• Decide the frequency of occurrence
• In some cases, if the occurrence of a minor-defect is
frequent in the code, it can be more severe. So from a
user's perspective, it is more serious even though it is a
minor defect.
• Isolate the defect
• Isolating the defect can help to find out its severity of
the impact.

2.11.2022. Doc. dr. sc. Ivan Vidović 69


Severity and priority example

2.11.2022. Doc. dr. sc. Ivan Vidović 70


Severity and priority example
• A very low severity with a high priority
• A logo error for any shipment website, can be of low
severity as it not going to affect the functionality of the
website but can be of high priority as you don't want any
further shipment to proceed with the wrong logo.
• A very high severity with a low priority:
• Likewise, for flight operating website, a defect in
reservation functionality may be of high severity but can
be a low priority as it can be scheduled to release in a
next cycle.

2.11.2022. Doc. dr. sc. Ivan Vidović 71


Guidelines for selecting severity
• Severity parameter is assessed by the tester whereas
the priority parameter is assessed by the product
manager or by the triage team.
• For prioritizing the defect, it is imperative for a tester to
choose the right severity to avoid confusion with the
development team.
• Guidelines:
• Understand the concept of priority and severity well
• Always assign the severity level based on the issue type as
this will affect its priority
• Understand how a particular scenario or Test Case would
affect the end-user
• Need to consider how much time it would take to fix the
defect based on its complexity and time to verify the defect

2.11.2022. Doc. dr. sc. Ivan Vidović 72


Psychology of testing

2.11.2022. Doc. dr. sc. Ivan Vidović 73


Role of software tester
• Identify the different end users of the system and
their interaction with the same.
• Capture the important scenarios for each end user.
It’s good to note that we need to capture the story
around the scenario and not just steps.
• Talk to different stakeholders including the
customers (in case if you have) on how the feature
might be used and capture the scenarios.
• Plan towards uncovering the critical issues of the
system as early as possible.
2.11.2022. Doc. dr. sc. Ivan Vidović 74
Objective of a software tester
• Find bugs as early as possible and make sure they
get fixed.
• To understand the application well.
• Study the functionality in detail to find where the
bugs are likely to occur.
• Study the code to ensure that each and every line
of code is tested.
• Create test cases in such a way that testing is done
to uncover the hidden bugs and also ensure that
the software is usable and reliable.

2.11.2022. Doc. dr. sc. Ivan Vidović 75


Mindset of tester
• A professional tester approaches a product with
the mindset that the product is already broken – it
has bugs and it is their job to find them out.
• A good professional tester:
• Takes nothing at face value.
• Always asks the question „why”?
• Seek to drive out certainty where there is none.
• Seek to illuminate the darker part of the projects with
the light of inquiry.

2.11.2022. Doc. dr. sc. Ivan Vidović 76


Who wants to be a tester?
• A destructive process
• Bring bad news („your baby is ugly”)
• Under worst time pressure (at the end)
• Need to take a different view, a different mindset
(„What if it isn’t?”, „What could go wrong?”)
• How should fault information be communicated (to
authors and managers)?

2.11.2022. Doc. dr. sc. Ivan Vidović 77


Psychology of testing
• Testing is a destructive process:
• Finding errors in a software
• Start testing with the assumption that the program
contains errors
• A successful test case is one that finds an error.
• It is difficult for a programmer to test his/her own
program effectively with the proper frame of mind
required for testing.

2.11.2022. Doc. dr. sc. Ivan Vidović 78


Psychology of testing
1. Software testing is the process to prove that the software
works correctly
• The person’s intentions would mostly revolve around the point to prove
the software works. He/She will only give those inputs for which
correct results are obtained.
• The above explanation is the typical psychology of testing by a software
developer.
2. Testing is the process to prove that the software doesn’t work
• This type of psychology would bring out the most of the defects in the
software. A software tester should therefore have such a psychology to
push the software beyond its boundaries.
3. Testing is the process to detect the defects and minimize the
risks associated with the residual defects
• Practically, if at any point, the software product development starts,
the testing process should start and keep in track the number of bugs
being detected while correcting them.

2.11.2022. Doc. dr. sc. Ivan Vidović 79


Psychology of testing
• Psychological factors that influence testing and its
success include clear objectives for testing:
• The proper roles and balance of self-testing and
independent testing
• Clear, courteous communication and feedback on
defects
• Also contrast mindset of a tester and of a developer

2.11.2022. Doc. dr. sc. Ivan Vidović 80


Code of ethics

2.11.2022. Doc. dr. sc. Ivan Vidović 81


Code of ethics
• All the testers are expected to follow certain code
of ethics after getting certified by the certification
board.
• These ethics makes them a better and responsible
tester.
• There are eight commandments which need to be
followed to attain the level of a certified tester.

2.11.2022. Doc. dr. sc. Ivan Vidović 82


Code of ethics
1. Public
Certified software testers shall act consistently with
the public interest.

• Certified software testers shall consider the wider


public interest in their actions.

2.11.2022. Doc. dr. sc. Ivan Vidović 83


Code of ethics
2. Client and Employer
Certified software testers shall act in a manner that is
in the best interest of their client and employer,
consistent with the public interest.

• Certified software testers are to treat any


information with care and attention, and act
responsibly to the owner(s) of this information,
employers and the wider public interest.

2.11.2022. Doc. dr. sc. Ivan Vidović 84


Code of ethics
3. Product
Certified software testers shall ensure that the
deliverables they provide (on the products and
systems they test) meet the highest professional
standards possible.

• Certified software testers are supposed to provide


the best quality deliverables to the end-users,
employers and the clients.

2.11.2022. Doc. dr. sc. Ivan Vidović 85


Code of ethics
4. Judgment
Certified software testers shall maintain integrity and
independence in their professional judgment.

• Certified software testers shall always maintain


independent judgment in terms of understanding
the requirements.

2.11.2022. Doc. dr. sc. Ivan Vidović 86


Code of ethics
5. Management
Certified software test managers and leaders shall
subscribe to and promote on ethical approach to the
management of software testing.

• Certified software testers shall always promote and


follow the ethical approach for software test
management process and should not by-pass the
process flow.

2.11.2022. Doc. dr. sc. Ivan Vidović 87


Code of ethics
6. Profession
Certified software testers shall advance the integrity
and reputation of the profession consistent with the
public interest.

• Certified software testers should act professionally


keeping their personal interest aside and do justice
with the reputation of the profession.

2.11.2022. Doc. dr. sc. Ivan Vidović 88


Code of ethics
7. Colleagues
Certified software testers shall be fair to and
supportive of their colleagues and promote
cooperation with software developers.

• Certified software testers shall always be


professional with software developers and their
colleagues. They can only be able to deliver a
quality product when they join hands.

2.11.2022. Doc. dr. sc. Ivan Vidović 89


Code of ethics
8. Self
Certified software testers shall participate in lifelong
learning regarding the practice of their profession
and shall promote an ethical approach to the
practice of the profession.

• Certified software testers shall practice continuous


learning and promote the ethical way of doing the
testing.

2.11.2022. Doc. dr. sc. Ivan Vidović 90

You might also like