5/6/2020

 Definitions – Audit, Audit Criteria,

Audit Evidences

1. Audit  Types of Audits

Basics  Purposes and Benefits of Auditing

 Audit Participants
 Independence and Objectivity

Audit
Systematic, independent and
documented process for
obtaining audit evidence and
evaluating it objectively to
determine the extent to which
the audit criteria are fulfilled.

1
 5/6/2020

Audit
 Systematic,
 independent and
 documented process
for
 obtaining audit evidence2 and
 evaluating it objectively
 to determine the extent to which the
audit criteria1 are fulfilled.

Audit Criteria
Set of policies, procedures or
requirements used as a
reference against which the
audit evidence is compared.

2
 5/6/2020

Audit Criteria
Set of policies,
procedures or
requirements
 used as a reference
 against which the audit
evidence2 is compared.

Audit Evidence
Records, statement of facts or
other information which are
relevant to the audit criteria
and verifiable.

3
 5/6/2020

Audit Evidence
Records,
statement of facts or
other information

 which are relevant to the

audit criteria1 and
verifiable.

 Definitions – Audit, Audit Criteria,

Audit Evidences

1. Audit  Types of Audits

Basics  Purposes and Benefits of Auditing

 Audit Participants
 Independence and Objectivity

4
 5/6/2020

Types of Audits

Product Audit First Party Audit

Process Audit Second Party Audit

System Audit Third Party Audit

Product Audit
Assessment of “fitness for
Product Audit use”
Products meet the design
Process Audit requirements

System Audit

10

5
 5/6/2020

Process Audit
 One specific process, activity or
function
Product Audit
 To compare the actual process
with the documented
Process Audit requirements of the process.

Inputs Process Output

System Audit

Step 1 Step 2 Step 3

11

System Audit
A comprehensive audit of
Product Audit multiple processes
Includes the interaction
Process Audit between processes

System Audit

12

6
 5/6/2020

Product, Process and System Audit

System

Process

Product

13

First Party Audit

First Party Audit
Second Party Audit
Third Party Audit
Internal audits
Performed within an
organization
Auditors have no vested
interest in the area being
audited

Suppliers Organization Customers

14

7
 5/6/2020

Second Party Audit

Performed by Customers on
First Party Audit suppliers
Before or after awarding a
Second Party Audit contract

Third Party Audit

Suppliers Organization Customers

15

Third Party Audit

Performed by an audit
First Party Audit organization independent of
the customer-supplier
relationship
Second Party Audit
Free from any conflict of
interest
Third Party Audit Third Party

Suppliers Organization Customers

16

8
 5/6/2020

First, Second and Third Party Audits

First Party • Internal

Second Party • By Client

• By a third party appointed by

Third Party client

17

Internal and External Audits

Internal External
Audit Audit

Second-party Audit

First-party Audit

Third-party Audit
18

9
 5/6/2020

Certification Audit
The most common
certification audit (quality) is
ISO 9001.
ISO does not conduct these
audits.
Certification audits are
conducted by Certification
Bodies (CB).

19

Certification Audit
Certification Bodies (CB) are
accredited by a member of
International Accreditation
Forum (IAF) Member.
Many countries have formed
accreditation bodies to
authorize ("accredit") the
certification bodies.

20

10
 5/6/2020

Certification Audit

ISO 9001 Organization

21

Certification Audit
A typical ISO 9001 certificate
will have the logo of the
Certification Body, the
Accreditation Body and the
International Accreditation
Forum (IAF) logo.

Types of Audits

22

11
 5/6/2020

 Definitions – Audit, Audit Criteria,

Audit Evidences

1. Audit  Types of Audits

Basics  Purposes and Benefits of Auditing

 Audit Participants
 Independence and Objectivity

23

Purposes and Benefits of Auditing

COMPLIANCE IMPROVEMENT

Ensures compliance to the industry, Identifies opportunities for

and statutory/regulatory improvement
requirements Reduce rework, rejections
Avoid lawsuits Reduced cost
Maintain market standing and/or Increase sales
reputation
Find the problems before the
Compliance to procedures and customer does
internal work processes
Corrective actions are effective
Provides confidence to stakeholders

24

12
 5/6/2020

 Definitions – Audit, Audit Criteria,

Audit Evidences

1. Audit  Types of Audits

Basics  Purposes and Benefits of Auditing

 Audit Participants
 Independence and Objectivity

25

Audit Participants
Client – organization or person
requesting an audit.

Auditor – A person who

conducts an audit

Auditee – organization or
individual being audited

26

13
 5/6/2020

Client - Responsibilities
Initiates audit
Determines audit purpose
and scope
Provide resources
Receives the audit report
Determine the report
distribution

27

Auditor - Responsibilities
 Understand the purpose, scope and
audit criteria.
 Plans the audit
 Perform the audit
 Collect audit evidences
 Analyze audit evidences
 Reports the audit
 Follows up the action on audit
findings

28

14
 5/6/2020

Lead Auditor - Responsibilities

 Balance the strength and
weaknesses of team members
 Manage the audit process
 Represent the audit team
 Lead the audit team
 Prepare and complete the
audit report

29

Auditee - Responsibilities
 Inform the staff
 Provide resources (interview
room, communications and
clerical support)
 Assign a guide for the audit team
 Show objective evidence
 Co-operate
 Determine and initiate corrective
actions

30

15
 5/6/2020

Audit Participants - 2
Technical Expert – a person who
provides specific knowledge or
expertise to the audit team.

Observer – a person who

accompanies the audit team but
does not audit.

Guide – a person appointed by the

auditee to assist the audit team.

31

 Definitions – Audit, Audit Criteria,

Audit Evidences

1. Audit  Types of Audits

Basics  Purposes and Benefits of Auditing

 Audit Participants
 Independence and Objectivity

32

16
 5/6/2020

Independence
 Auditors are independent when they
render impartial and unbiased judgment
in the conduct of an audit.
 Conflict of interest is a situation in which
an internal auditor has a competing
professional or personal interest.
 Audit independence is essentially a state
of mind. Internal auditors can not be
physically independent of the
organization they are working for, but
they can always stay objective.

33

Objectivity
 Objectivity is a mental attitude that
auditors should maintain while
performing engagements.
 The auditor should have an impartial,
unbiased attitude and avoid conflict of
interest situations.
 auditors are not to accept fees, gifts, or
entertainment that may create the
appearance that the auditor's objectivity
has been impaired.

34

17
 5/6/2020

 Overview of Auditing Process

 Audit Purpose, Scope and Criteria
2. Audit  Audit Program and Plan

Planning  Auditor Selection

 Auditing Strategies
 Activities Prior to the Site Visit

35

1. Planning and Preparation

2a. Opening Meeting

Overview of 2b. Audit Interviews

Auditing
Process 2c. Closing Meeting

3. Reporting

4. Follow-up and Closure

36

18
 5/6/2020

 Overview of Auditing Process

 Audit Purpose, Scope and Criteria
2. Audit  Audit Program and Plan

Planning  Auditor Selection

 Auditing Strategies
 Activities Prior to the Site Visit

37

Audit Purpose, Scope and Criteria

Audit Purpose
What is to be accomplished by the
audit?
Defined by the audit client.

Audit Scope
Extent and boundaries of the audit,
Satisfies the purpose of the audit

Audit Criteria
Reference against which conformity is
determined

38

19
 5/6/2020

Audit Purpose
Two main purposes of an audit are:
Compliance
Improvement
Examples of audit purpose/ objective:
Conformity of the management system
Meeting relevant statutory and regulatory
requirements and other requirements to
which the organization is committed
Effectiveness of the management system
in meeting its intended results
Identifying opportunities for improvement

39

Audit Scope
Extent and boundaries of an audit
Clearly defining the audit scope is
important in determining the
budget, human resources, and
time required for the audit
The scope defines what is included
and what is excluded:
Location (Plant A only)
Functions (Materials Management)
Processes / Activities (From receiving a
material requisition to receipt of
material)

40

20
 5/6/2020

Audit Criteria
Set of policies, procedures or
requirements used as a reference
against which audit evidences are
compared.
Examples of Audit Criteria include:
National or international standards
Industry codes and standards
Laws and regulations
Contracts
Purchase orders
Customer specifications

41

Audit Criteria
 ISO 9000:2015
Quality management systems - Fundamentals
and vocabulary
 ISO 9001:2015
Quality management systems Requirements
 ISO 9004:2018
Quality management - Quality of an
organization - Guidance to achieve sustained
success
 ISO 19011:2018
Guidelines for auditing management systems

42

21
 5/6/2020

Audit Criteria – Management System

Standards
 ISO 9001 – Quality management system
 ISO 14001 – Environmental management
system
 ISO 22000 - Food safety management
systems
 ISO 22301 - Business continuity
management systems
 ISO 45001 - Occupational health and
safety management systems
 ISO 50001 - Energy management systems

43

Audit Criteria – Industry Specific

Management System Standards
ISO 13485 – Medical devices
ISO 17025 - Competence of testing
and calibration laboratories
ISO 29001 - Petroleum,
petrochemical and natural gas
industries
TL 9000 – Telecommunications
AS 9100 – Aerospace
IATF 16949 - Automotive

44

22
 5/6/2020

 Overview of Auditing Process

 Audit Purpose, Scope and Criteria
2. Audit  Audit Program and Plan

Planning  Auditor Selection

 Auditing Strategies
 Activities Prior to the Site Visit

45

Audit Program
Audits are planned and
documented
Formal and systematic
Are never informal

An audit program shall be planned,

taking into consideration
the status of the processes
importance of the processes
the results of previous audits

Frequency also depends on

Criticality of product or service

46

23
 5/6/2020

Audit Plan
Lead Auditor prepares the plan
Communicate plan to
Client
Auditee
Other Stakeholders

47

Audit Plan
Audit planning should address or
reference the following:
Audit objectives
Audit scope … functions, and processes
to be audited
Audit criteria … ISO 9001:2015
Locations, dates, expected time and
duration of the audit
Audit team members

48

24
 5/6/2020

Audit Plan
Formal audit notification required
for second-party or third-party
audit.
Generally, advance notification is
provided.
Communicate the plan to:
Client
Auditee
Other stakeholders as applicable

49

 Overview of Auditing Process

 Audit Purpose, Scope and Criteria
2. Audit  Audit Program and Plan

Planning  Auditor Selection

 Auditing Strategies
 Activities Prior to the Site Visit

50

25
 5/6/2020

Audit Team Selection

The number and composition of
the auditor team depend upon:
Objective, Scope, and Audit Criteria
Competence of team members
Teamwork, and the ability to interact
effectively with the auditee
statutory, regulatory, contractual and
accreditation/certification requirements
To be continued …

51

Audit Team Selection

The number and composition of
the auditor team depend upon:
Language / cultural issues
Location of audit
Cost consideration
Time available

52

26
 5/6/2020

Auditor Competencies
Factors to consider:
Auditing Knowledge
Audit principles, procedures and methods
Management system
Technical Knowledge
Contractual requirements
Codes and standards
Discipline / Sector-specific
Personal Behavior

53

Auditor Competencies (Positive)

Ethical
Open-minded
Diplomatic
Tenacious (persistent)
Decisive
Self-reliant
Culturally sensitive
Collaborative

54

27
 5/6/2020

Auditor Competencies (Negative)

Argumentative
Opinionated
Aggressive
Inconsiderate
Inflexibility
Lazy
Impractical
“Know-it-all”
Indecisive

55

What Good Auditor Do?

Plan and organize
Time management
Prioritize and focus on significant
issues
Interview, listen, observe and
review documents
Communicate effectively
Understand the risks associated

56

28
 5/6/2020

 Overview of Auditing Process

 Audit Purpose, Scope and Criteria
2. Audit  Audit Program and Plan

Planning  Auditor Selection

 Auditing Strategies
 Activities Prior to the Site Visit

57

Auditing Strategies
Trace forward
Trace backward
Random selection

58

29
 5/6/2020

Auditing Strategies – Trace Forward

An examination from beginning to
the end
Beneficial to get the whole picture
from the start to finish

59

Auditing Strategies – Trace Backward

Beginning at the end and working
back through the process
Understanding of end objective is
attained right away
All product records exist

60

30
 5/6/2020

Auditing Strategies – Random Selection

Advantage:
Where time and personnel are limited
Most frequently used
Use of flowchart to identify important
steps
Flexible and saves time
Disadvantage:
Additional note-taking
Difficulty in understanding the process
flow
Experienced auditor required

61

Horizontal vs. Vertical Audit

Element Audit (Horizontal)
Selected elements are audited across
multiple departments (e.g. training)
To satisfy elements on checklist several
departments audited
Department Audit (Vertical)
Several system elements are audited in a
single department (e.g. design
department)
Saves times (+)
Judges company on management of one
or selected departments (-)
The auditor should be very familiar with
the requirements

62

31
 5/6/2020

Horizontal vs. Vertical Audit

Sales Design Production Dispatch After Sales
4. Context of the organization
5. Leadership
6. Planning
7. Support
8. Operation
9. Performance evaluation
10. Improvement

63

 Overview of Auditing Process

 Audit Purpose, Scope and Criteria
2. Audit  Audit Program and Plan

Planning  Auditor Selection

 Auditing Strategies
 Activities Prior to the Site Visit

64

32
 5/6/2020

Activities Prior to the Site Visit

Notify the auditee

Review auditee documents
Arrange audit logistics
Prepare the audit checklist

65

Notify the auditee

Advance/formal notice to the
auditee
Notification to auditee Quality
Manager and the top management

66

33
 5/6/2020

Notify the auditee

Contents of audit notification:
Audit objective, scope and criteria
Names of the audit team members and
the team leader
Official contacts of lead auditor
Time and location of the opening
meeting
Identification of areas of special concern
Tentative audit schedule (to be
reconfirmed in the opening meeting)
Documents to be submitted prior to the
audit for review.

67

Activities Prior to the Site Visit

Notify the auditee

Review auditee documents
Arrange audit logistics
Prepare the audit checklist

68

34
 5/6/2020

Typical Quality System Documents

Quality Manual
Procedures Quality Manual
Work Instructions
Procedures
Forms
Work Instructions

Forms

69

Quality Manual
ISO 9001:2015 does not require a
Quality Manual
Manual based on ISO 9001
standard vs manual based on
processes.

70

35
 5/6/2020

Procedures
Procedures provide a high-level
overview of the process
Procedures do not include the
“detailed how” component of the
process
Procedures are generally multi-
discipline

71

Work Instructions
Step by step instructions, how the
work is done.

72

36
 5/6/2020

Forms
To record the compliance
The terms documents and records
have now been changed to
“documented information” in ISO
9001:2015

73

Other Types of Documents

Product specification
Drawings
Inspection and Test Plan
Inspection and Test Reports
Results of prior reviews and audits

74

37
 5/6/2020

Pre-audit Information
Most organizations have electronic
copies of documentation and the
information can be shared
electronically.
Auditee might not want to share a
copy of some confidential
documents.
At this stage, the audit team needs
limited information for the
preparation purpose only.

75

Review auditee documents

Review documents related to the
auditee organization
Contract, Specifications,
Quality Manual,
Procedures, Guidelines
Organization Charts
Codes and Regulations etc.
Previous audit reports/programs

76

38
 5/6/2020

Review auditee documents

Advantages of in-office review
Gain a good understanding of the
auditee
Enable the team to prepare relevant
questions to the auditee during the
opening meeting.
Help the team to understand well the
criteria (what is to be done; by whom
and when).
Saves the audit investigation time

77

Review auditee documents

Identifying areas for audit
emphasis (risk area). These
include:-
Areas identified/documented in the Risk
Register.
Areas of the high possibility of
mismanagement
Areas of large volumes of transactions
Areas of concerns identified by Client
Major weakness/deficiencies identified
in the prior audits

78

39
 5/6/2020

Activities Prior to the Site Visit

Notify the auditee

Review auditee documents
Arrange audit logistics
Prepare the audit checklist

79

Arrange for Audit Logistics

Travel and accommodation
Safety and security considerations
Personal Protective Equipment (PPE)
Location and/or Camera Permit
Need for a Guide
Translators
Facilities
Working area, conference room,
internet, printer, tea/coffee and working
lunch

80

40
 5/6/2020

Activities Prior to the Site Visit

Notify the auditee

Review auditee documents
Arrange audit logistics
Prepare the audit checklist

81

Audit Checklist - Purpose

Memory tool
To ensure that all areas have been
covered
Help in time management
Helps in note-taking
The checklists should be a good
servant, never the master, of the
auditor.
Evidence of audit effectiveness

82

41
 5/6/2020

Audit Checklist
Developed by Lead auditor or
auditor
Based on audit purpose and scope
Include open-ended questions
Should have space to record
response
Send it to auditee prior to the
audit, along with the audit
notification

83

Audit Checklist - Generic vs Specific

Generic checklists
Supplement with specific items

 Specific Checklists
Prepared for a specific use.

84

42
 5/6/2020

Audit Checklist – Scoring vs. Non-scoring

Scoring checklists:
Used for comparison purpose
The score may become the goal, and
bias in marking

Non-scoring checklists:
Good for continuous improvement and
are flexible

85

Audit Checklist – Advantages

Promotes planning for the audit
Thorough coverage of the scope
Time management during the
audit
Consistent audit approach
Serves as a memory aid
Objective evidence that the audit
was performed
Information base for future audits

86

43
 5/6/2020

Audit Checklist – Disadvantages

Questionnaires narrow the vision
Questionnaires may obstruct
communication
Too strict following of the checklist
may result in the omission of
important audit trails
Generic checklists may not add any
value

87

 Opening Meeting
Audit  Interviews and Data Collection

Conducting  Summarizing the Audit Findings

 Closing Meeting

88

44
 5/6/2020

Audit Opening Meeting Purpose

Introduce the audit team and their
roles;
Confirm the agreement of all
participants (e.g. auditee, audit
team) to the audit plan;
Ensure that all planned audit
activities can be performed.

89

Audit Opening Meeting Participants

Auditee senior management
Quality Manager
Audit guides
All audit team participates
Lead Auditor chairs the meeting

Don’t forget to keep a record of

meeting participants

90

45

Audit Opening Meeting Agenda
Confirmation of the following
items, as appropriate:
the audit objectives, scope and criteria;
the audit plan and other relevant
arrangements with the auditee, such as
the date and time for the closing
meeting, and any change(s) needed;
the language to be used during the
audit;
the availability of the resources and
facilities needed by the audit team;
matters relating to confidentiality and
information security;
91
Audit
Conducting
92
5/6/2020
relevant access, health and safety,
security, emergency and other
arrangements for the audit team;
activities on site that can impact the
conduct of the audit.
the method of reporting audit findings
including criteria for grading, if any;
conditions under which the audit may
be terminated;
any questions/clarifications?
 Opening Meeting
 Interviews and Data Collection
 Summarizing the Audit Findings
 Closing Meeting
46
 5/6/2020

Interviews and Data Collection

Interviewing
Observation / Measurement
Document and Record review

93

Interviewing Techniques
Establish suitable climate
Put auditee at ease
Ask questions in a conversational
manner
Questions should:
Yield the relevant information
Should not suggest answers
Should not contain emotional words

94

47
 5/6/2020

Three Types of Questions

Open-ended questions
Closed-ended questions
Clarifying questions

95

Open-ended Questions
Start with … What? Why? Where?
Who? When? How?
Advantage: Yield informative
answer
Limitation: May lead to
conversation get side-tracked!
One way to keep the audit on track
is: ….. Show me!

96

48
 5/6/2020

Closed-ended Questions
Closed-ended questions
answer: Yes/No
Open questions
answer: a few words and explanation
Intended to yield very specific
information
Disadvantages:
Do not bring much information
If used too often may create the
impression of cross-examination

97

Clarifying Questions
Intended to clarify, retrieve full
information and prevent
misunderstanding
Disadvantages:
If used too often may create impression
that you were not listening
Are time-consuming
If you are not prepared to listen in full
don’t ask them

98

49
 5/6/2020

Interviewing Tips
Generally use open-ended
questions and sparingly closed-
ended questions
Focus on the process and not the
individual
Take proper notes (drawing
number/rev. number, part number,
record reference)
Share potential findings before
leaving the interview

99

Interviews and Data Collection

Interviewing
Observation / Measurement
Document and Record review

100

50
 5/6/2020

Observations
Typical observations include:
What is it used for?
Does this need to be calibrated?
Was it calibrated?
Is there a record?
What is the reading?
Is the reading within the acceptable
range?
What if this is not in the acceptable
range?
Identification and traceability?
Storage location & conditions?

101

Observations
Make sure you are not obstructing
the work
Be aware of safety requirements
Consider taking pictures only if it is
permitted and agreed with the
auditee organization

Don’t forget to keep the record of

observations (part number,
location, machine number etc.)

102

51
 5/6/2020

Interviews and Data Collection

Interviewing
Observation / Measurement
Document and Record review

103

Document / Record Review

Ask for documents and records
during interviews and observations
Consider random sampling when
reviewing documents and records
Documented information can be in
any format and media and from
any source

Don’t forget to keep a record of

observations (part number,
location, machine number etc.)

104

52
 5/6/2020

Document / Record Review

Sampling
You do not have time to check
everything. Select representative
samples
Cover the relevant work period
when selecting
documents/records.

105

Documented Information (ISO 9001:2015)

Is it available and suitable for use,
where and when it is needed?
Is it is adequately protected (e.g.
from loss of confidentiality,
improper use, or loss of integrity)?
Is it preserved to ensure legibility?
Is change control (revision) in
place?
Is it protected from unintended
alterations?

106

53
 5/6/2020

Reliability of objective evidences

- Rules of Thumb
Documentary evidence is usually
better than verbal evidence.
Audit evidence is more reliable
when evidences obtained from
different sources are consistent.
(Corroboration)
Evidences generated through the
auditor's direct observation,
inspection, and computation are
usually better than evidence
obtained indirectly.

107

 Opening Meeting
Audit  Interviews and Data Collection

Conducting  Summarizing the Audit Findings

 Closing Meeting

108

54
 5/6/2020

Notes Taking
Make your notes:
Comprehensive
Accurate
Precise
Legible

109

Notes Taking
Documents:
Title and document number
Revision number
Issue date
Location where the document was seen
Part:
Part description
Identification number
Person:
Name
Title
Department

110

55
 5/6/2020

Corroboration
To strengthen with other evidence,
to make more certain.
More important for
data/information which could be
questionable/doubtful.
Confirming or verifying using
multiple sources.

111

 Opening Meeting
Audit  Interviews and Data Collection

Conducting  Summarizing the Audit Findings

 Closing Meeting

112

56
 5/6/2020

Prior to Audit Closing Meeting

Audit team meets before the
closing meeting to:
Confirm achievement of audit objective,
coverage of audit scope, and fulfilment
of audit criteria.
Review audit findings
Agree on audit conclusions
Prepare recommendations, if specified
by audit plan
Discuss audit follow-up

113

Audit Closing Meeting Purpose

To present the audit findings and
conclusions
To ensure a clear understanding of
the audit results
Agree on the timeframe for
corrective actions.

114

57
 5/6/2020

Audit Closing Meeting Participants

Auditee senior management
Quality Manager
People responsible for the
areas/functions audited
Audit guides
All audit team participates
Lead Auditor chairs the meeting

Don’t forget to keep a record of

meeting participants

115

Audit Closing Meeting Agenda

Agenda includes:
Audit purpose and scope
Explain about sampling
Method of reporting (grading)
Presentation of audit findings
The draft report and expected time of
the release of the final report
The expectation of action on audit
findings
Recommendations and opportunities for
improvement (only if specified by audit
objective)
Assure confidentiality of information
Any disagreement (resolve or else
record)

116

58
 5/6/2020

 Effective Audit Reporting

Audit  Non-Conformities Classification
Reporting  Corrective and Preventive Actions

117

Audit Report
A formal audit report needs to be
provided to the auditee.
The audit report might need the
approval of auditor management.
The report should be sent timely,
as agreed in the exit meeting.
The distribution of the report is
decided by the Client.

118

59
 5/6/2020

Audit Report Elements

 Auditee organization
 Purpose, scope and audit criteria
 Identification of the audit client
 Date, time and location of the audit
 Audit team members
 Opening and Closing meeting participants
 Area not covered (if any with justification)
 Audit findings, evidences and
classification
 Audit conclusion including a statement on
the degree to which the audit criteria
have been fulfilled

 Personnel interviewed  Any unresolved diverging opinions

between the audit team and the auditee
 Reference of documents reviewed
 A statement that the audits by nature are
a sampling exercise

119

Audit Report Elements

The audit report may also include:
Obstacles observed during the audit
which could reduce the audit reliability
Best practices observed
Action plan to resolve findings (if agreed
during the audit)
Opportunities for improvement (if it is a
part of the audit objective)

120

60
 5/6/2020

Audit Report – Avoid These

Acronyms
Name of individual employees
Nit-pick (trivial many)
Emotional or argument statements
Surprises
Technical jargon or too
complicated language
Not including positive aspects
Late report *

121

Audit Report – Timing

Complete as soon as possible
Problems with Formal Report
procrastination
The longer it is put off, the lesser
interest in pursuing the corrective
actions
A signal to the auditee management
that audit is not important as was
initially believed

122

61
 5/6/2020

 Effective Audit Reporting

Audit  Non-Conformities Classification
Reporting  Corrective and Preventive Actions

123

Nonconformities Classification
Nonconformities and their
supporting audit evidence should
be recorded.
Nonconformities could be graded
depending on the level of risk.
Typically third-party auditors
classify them as the minor or major
NCR (Non-conformance Report)*
Nonconformities identified should
be discussed with the audit team
during team meetings.

124

62
 5/6/2020

Nonconformities Classification
Impact (or Severity)
Nonconformities are classified as
major or minor based on: Very Low Medium High Very
Severity Low High
Frequency

Probability (or Frequency)

Level of risk Very Medium Medium High High High
High

High Low Medium Medium High High

Medium Low Medium Medium Medium High

Low Low Low Medium Medium Medium

Very Low Low Low Low Medium

Low

125

 Effective Audit Reporting

Audit  Non-Conformities Classification
Reporting  Corrective and Preventive Actions

126

63
 5/6/2020

Corrective and Preventive Actions

Correction
action to eliminate a detected
nonconformity
Corrective Action
action to eliminate the cause of a
nonconformity and to prevent
recurrence
Preventive Action
action to eliminate the cause of a
potential nonconformity or other
potential undesirable situation

127

Correction
action to eliminate a detected
nonconformity
A correction can be made in
advance of, in conjunction with or
after a corrective action.
A correction can be, for example,
rework or regrade.

128

64
 5/6/2020

Corrective Actions
action to eliminate the cause of a
nonconformity and to prevent
recurrence.
There can be more than one cause for a
nonconformity.
Corrective action is taken to prevent
recurrence whereas preventive action is
taken to prevent occurrence.

129

Corrective Actions
When a nonconformity is
identified in the audit report, the
auditee needs to take two actions:
Correction (remedial or
containment actions)
Corrective Action (to prevent
recurrence)

130

65
 5/6/2020

Corrective Actions – Key Steps

Identify the problem
Contain the problem,
Determine the causes of the
problem,
Propose solutions to eliminate
them or prevent their recurrence
Verify that the solutions are
implemented
Confirm their effectiveness.

131

Preventive Actions
ISO 9001:2015 does not have
requirements related to the
Preventive Action
The concept of Preventive Actions
is addressed using the “Risk-Based
Thinking” concept.

132

66
 5/6/2020

Audit  Audit Follow-up

Follow-up  Audit Closure
and Closure

133

Audit Follow-up and Closure

 The audit team identifies nonconformity.
 Auditee proposes Corrective Actions.
 The Client, the audit program manager or the
lead auditor review the proposed Corrective
Actions.
 The auditee complete the necessary actions
 The lead auditor verifies the completion and
effectiveness of Corrective Actions. *
 Follow up on ineffective Corrective Actions
 Close the audit

134

67
 5/6/2020

Audit Follow-up and Closure

 Follow-up audit to verify the Corrective Action
implementation and effectiveness.
 Effectiveness of the Corrective Action ensures that the
desired objectives are met.

 To save time and money, the verification could

be done via:
 Document review
 Remote audit
 Combined with subsequent audit

135

Audit Follow-up and Closure

Why would the reviewer reject the Corrective
Action Plan?
 Auditee does not understand the difference
between the Correction and Corrective Action.
 The root cause of the nonconformity is
identified and addressed. The focus is only on
symptoms.
 Typical list of causes which are not acceptable:
 Human error
 The procedure ignored or not followed.
 ……….
 ………

136

68
 5/6/2020

Audit Follow-up and Closure

When all agreed Corrective Actions
have been completed and found to
be effective, the audit can be
closed.
Audit records to be retained as per
procedure or applicable
requirements.

137

 Why to Sample?
Sampling  Types of Sampling
 Sampling Risk/Errors

138

69
 5/6/2020

 Because of the cost and time

involved in studying the entire
population.

Why Sampling?

139

 Probability Samples
 Everyone in the population has an
equal chance of being selected
 Non-Probability Samples
 Where the probability of selection
can't be accurately determined.
 Sample may not be (generally isn’t)
representative of the general
population
Types of
Sampling
140

70
 5/6/2020

Simple Random Sampling

Systematic Random Sampling

Probability

Stratified Random Sampling

Sampling

Cluster Sampling

Accidental / Convenience Sampling

Non Probability

Judgemental Sampling
Sampling

Quota Sampling Types of

Sampling
141

Simple Random Sampling

 Each item in the population has an equal

chance of being selected.
 Examples: Using random tables, Random
draw of lot (lottery)

Simple Random Sampling

Systematic Random Sampling

Types of
Probability

Stratified Random Sampling

Sampling

Cluster Sampling Sampling

142

71
 5/6/2020

Systematic Random Sampling

 Select elements at regular intervals through

that ordered list.
 Example: Checking every 6th piece produced
by the machine.

Simple Random Sampling

Systematic Random Sampling

Types of
Probability

Stratified Random Sampling

Sampling

Cluster Sampling Sampling

143

Stratified Random Sampling

 Used to ensure that sub-groups within a

population are represented proportionally
in the sample.
 Example: If 10 people are drawn to
represent a country, 5 of them are male and
5 females to avoid the sex bias.
Simple Random Sampling

Systematic Random Sampling

Types of
Probability

Stratified Random Sampling

Sampling

Cluster Sampling Sampling

144

72
 5/6/2020

Cluster Sampling
 Sometimes it is more cost-effective to select
respondents in groups ('clusters'). Sampling
is often clustered by geography, or by time
periods.
 Example: Survey all customers visiting
particular stores on particular days.
Simple Random Sampling

Systematic Random Sampling

Types of
Probability

Stratified Random Sampling

Sampling

Cluster Sampling Sampling

145

Convenience Sampling
 The researcher selects whom ever is
convenient. The samples are being drawn
from that part of the population which is
close to hand
 Example: A researcher at the mall selects
the first five people who walk by to get
their opinion of a product.

Accidental / Convenience Sampling

Non Probability

Judgemental Sampling Types of

Sampling

Quota Sampling Sampling

146

73
 5/6/2020

Judgmental Sampling
 The researcher chooses the sample based
on who they think would be appropriate for
the study.
 Example: Auditor selects a sample based on
the concerns he/she had in the earlier audit

Accidental / Convenience Sampling

Non Probability

Judgemental Sampling Types of

Sampling

Quota Sampling Sampling

147

Quota Sampling
 A quota is established and auditor are free
to choose any sample they wish as long as
the quota is met.
 Example: 2% of the calibration records.

Accidental / Convenience Sampling

Non Probability

Judgemental Sampling Types of

Sampling

Quota Sampling Sampling

148

74
 5/6/2020

1000 pieces
Accept the lot if 3 or less
Check 80 pieces
are defective
Reject the lot if 4 or more
are defective
Standards
Attribute Sampling MIL-STD-105 (withdrawn)
ANSI/ASQ Z1.4
Pass/Fail Dodge-Romig

Variable Sampling MIL-STD-414 (withdrawn)

ANSI/ASQ Z1.9
Dimension

Acceptance Sampling Standards Slide 149

149

Attribute Sampling
 You need to decide
 Level (e.g. I, II, III, S1, S2, S3 or S4)
 AQL (Acceptable Quality Limit) – e.g. 1.5%
 Single, Double or Multiple Sampling Plan
 Reduced, Normal or Tightened inspection

Acceptance
Sampling Plans

150

75
 5/6/2020

Attribute Sampling
Example
• Lot size : 1,000, General inspection level
II
• Acceptable Quality Limit(AQL): 1.5%
• Take 80 random samples:
• Accept the lot if 3 or less are
rejected.
• Reject the lot if 4 or more rejected.

151

Actual Lot Condition

Good Lot Bad Lot

Correct Type II Error

Accept the Lot
Conclusion Conclusion (Buyer’s Risk)
Based on
Sampling Type I Error Correct
Reject the Lot
(Producer’s Risk) Conclusion

Sampling Terms Slide 152

152

76
 5/6/2020

Level of Significance
Type I error (alpha) Type II error (beta)
or Type I Error:
Name Producer’s risk/ Consumer’s risk
α = 1 – C (0.10, 0.05, Significance level
0.01) 1 minus error is
called
Confidence level Power of the test

Example of Fire False fire alarm leading Missed fire leading to

Alarm to inconvenience disaster
Confidence Level: Effects on Unnecessary cost Defects may be produced
C = 0.90, 0.95, 0.99 process increase due to frequent
changes
(90%, 95%, 99%) Simple definition Innocent declared as Guilty declared as innocent
guilty

Sampling Terms Slide 153

153

77