Risk Assessment Tools For IVD Assays

Risk Assessment Tools for Identifying Hazards and Evaluating Risks Associated with IVD Assays
Robert C. Menson, PhD

AACC Expert Access Live on Line 06 July 2004
What Risks Must Be Managed?

Risk to safety of patients, users, handlers
Business
Regulatory
Product liability
2004 Menson & Associates, Inc AACC Expert Access Live on Line 2
Elements of the Risk Management Process

Risk Analysis Risk Evaluation Risk Control Post-Production Information
Assessment
Management
Risk Assessment Tools

Risk Matrix PHA= Preliminary Hazard Analysis FTA=Fault Tree Analysis FME(C)A=Failure Mode Effects (Criticality) Analysis HAZOP=Hazard Operability Analysis HACCP=Hazard Analysis and Critical Control Point
Applications of Risk Analysis

Design Production Pre-market Notifications Complaints MDR Change Control Failure Analysis Etc.
Risk Management
Life Cycle
Concept & Feasibility Development Scale-Up & Transfer Production
System Level Assessment Design Assessment Process Assessment Customer Feedback Assessment
2004 Menson & Associates, Inc
AACC Expert Access Live on Line
Risk Management
& Design Control
Concept & Feasibility
Planning
Development
Scale-Up & Transfer

Test Methods & Results
Production
Design Control Risk Assessment
Requirements
Plan
Specifications
Productions Methods Risk Management Report
Change Records
Preliminary Hazard Analysis
Risk Management Plan
Detailed Analysis (FMEA, FTA, HACCP, HAZOP)
Risk Reviews
Key Concepts of Risk

The frequency of the potential harm;
How often the loss may occur;
The consequences of that loss;

How large the loss might be;
The perception of the loss;

How seriously the stakeholders view the risk that might affect them.
Risk Management Terms

Intended Use/Purpose Harm Hazard Risk Residual Risk Tolerable Risk
Intended Use/Purpose
Use of a Product, Process or Service in accordance with the specifications, instructions and information provided by the manufacturer
ANSI/AAMI/ISO 14971:2000, definition 2.5
10
Harm
Physical injury or damage to health of people, or damage to property or the environment.
ISO/IEC Guide 51:1999, definition 3.3 Guidelines for inclusion of safety aspects in standards.
Hazard
Potential source of Harm
ISO/IEC Guide 51:1999, definition 3.5

Risk
Combination of the probability of occurrence of harm and the severity of harm.

Residual Risk
Risk remaining after protective measures have been taken.

Tolerable Risk
Risk which is accepted in a given context based on the current values of society.

15
Hazard Identification
Laws Codes Standards
Hazard Identifcation
Events MDRs Accidents Etc
List of Hazards
Brainstorming PHA FTA FMEA
16
Risk Matrix
Frequency Negligible Frequent Probable Occasional Remote
Severity
Minor Major Severe
L L T T
I I I L
H H I I
H H H I
17
Severity of Consequence Codes

Severity
Severe
Nuclear
Virtually complete loss of plant. -Many fatalities Extensive damage to plant. Few fatalities
Medical
Potential death or Permanent injury
Generic
Product must be scrapped. Loss of customer or repeat business. Compliance in jeopardy. Major service or downtime. Significant rework. Customer will notice. May cause injury. Loss of customer confidence Reduced service. Minor rework or re-inspection. Customer may notice. Minor nuisance. No effect on service. Customer wont notice.
Major
Potential serious injury. Not permanent
Minor
Negligible
Significant damage to plant. Severe injury or occupational illness Minor systems damage. Minor injury or occupational illness
Potential minor injury. Not permanent Potential minor discomfort. Not permanent.
18
Frequency Codes
Frequency
Frequent Probable Occasional Remote Improbable Incredible
Nuclear
> Once/yr 0.1-1/yr 0.01-0.1/yr 0.0001-0.01/yr 0.000001-0.0001/yr <0.000001/yr <1 in 1,000,000 yrs
Medical
>50% of time 10-50% of time 1-9.9% of time 0.1-0.99% of time 0.0001-0.1% of time <.0001% of time <1 in 10,000
Generic
Continual occurrences Occurrences are frequent Happens, probable reoccurrence Isolate occurrences happen Isolated occurrences possible Dont expect reoccurrence Not likely Incredible
19
Estimation of Risk Codes

H: High I: Intermediate L: Low T: Trivial Risk must be reduced Reduced to ALARP-cost a minor factor Reduce to ALARPconsider cost/benefit Broadly acceptable
ALARP=As Low As Reasonably Possible

PHA
Hazards Arising From Functional Equipment Failure Hazard System failure due to deterioration over life cycle Pump, Valve or Liquid Sensor Malfunction Software bug/failure Loss of bits on SRAM due to weak battery Installation of Incorrect Software Version Investigation/Controls Stress/Simulated Life Cycle Testing 8-Hour burn in Readily apparent to operator Sample can be reprocessed Backup sample Software Qualification Testing Backup Sample Battery life certification Drain testing Preventive maintenance QSR Training SOPs Internal Software Checks Sev. N Mi Freq. R R Impa ct
T L I T I
Mi Mi
O R
Ma
21
ALARP Curve
Increasing Probability of Occurrence
Intolerable Region
ALARP Broadly Acceptable Region
Maximum Tolerable Risk
Increasing Severity of Harm

FMEA vs FTA
FMEA Assumes component or part failure Identifies functional failure as a result of part failure FTA Assumes failure of the functionality of a product Identifies part/module failure as cause of functional failure
23
FMEA vs. FTA

FMEA Done for entire design Systematic way to predict new problems A bottoms-up analysis People expect the same results from FTA which is not true FTA Too difficult to do for entire design Systematic way to predict causes for problems A top down analysis People do not expect the same results from and FMEA Often a fault tree is used for a problem or an accident.
24
FTA
Assumes fault and analyzes possible causes Connection tool for PHA* to subsystems or modules Top down Deductive Evaluate system (or subsystem) failures Considered more structured than FMEA Graphical presentation--visual picture
*Preliminary Hazard Analysis
Failures
Primary--Due to internal causes that include poor design or use of inappropriate materials Secondary--Due to failures in the operation that include equipment failure Control--Due to failures in the systems that are in place to protect the quality and safety
e.g. raw material outside specification failure of safety switch failure of test method
FTA Basic Symbols

Basic Flow
FAULT
Fault in a box indicates that it is a result of subsequent faults Connects a preceding fault with a subsequent fault that could cause a failure Connects two or more faults that must occur simultaneously to cause the preceding fault
AACC Expert Access Live on Line 27
OR
AND
FTA During Design

P O O R F IT IN H A N D A S P E C T R A T IO T O O H IG H OR M A JO R D IA M E T E R TO O LARG E
Competing Design Criteria
A S P E C T R A T IO TO SM ALL BASE FLATNESS B A S E S IZ E TO O SM ALL R IM T O F IL L IN S U F F C IE N T
DOES NOT R E S T R IC T S P IL L A G E
OR
28
FTA Lab Failure

Other Outliers Patient Death
OR
Calibration Error
Lab Error
OR
Systematic
OR
Interference
Other
Random
29
FTA During Reliability

HAZARD
SYSTEM FAILURE
OR
4. x 10-9
DRIFT> LIMIT
AND
OR
1. x 10-16
4. x 10-9
CMPT A FAILS 5. x 10
-9
CMPT B FAILS 2. X 10 -8
CMPT C DRIFTS
REFERENCE DRIFTS 1. x 10 -9
3. x 10-9
30
What is FMEA?
FMEA
Failure Mode and Effects Analysis
FMECA
Failure Mode Effects and Criticality Analysis
31
What is FMEA?
Powerful prioritization tool Inductive High effective tool for identifying critical quality attributes High structured Methodical Breaks large complex designs into manageable steps
FMEA
Bottom up approach Evaluates specific failures Detailed analysis tool
Use in conjunction with PHA and FTA
Complements FTA
May lead to different failure results
33
Advantages of FMEA
Less analyst dependent than FTA Allows direct criticality assessment of components Valuable troubleshooting aid Identifies areas of weak design Identifies areas of high risk Prevention planning Identifies change requirements
Disadvantages of FMEA
Does not consider operator error Tedious May not apply to all systems--especially software May require extensive testing to gain information May miss some failure modes
Time pressures Information missing
Definitions
Criticality -- Weighting of hazard severity with the probability of failure Severity -- Seriousness of effect through its impact of the system function Occurrence -- Likelihood a specific failure will be caused by a specific cause under current controls, i.e. the probability Verification -- Ability of the current evaluation technique to detect potential failure during design Detection -- Ability of the current manufacturing controls to detect potential failure before shipping
Definitions
Risk Priority Number (RPN)
= (S) x (O) x (D) or (V)
Severity (S) Likelihood of occurrence (O) Likelihood of detection (D) Likelihood of verification (V)
37
FMEA Example
P O T E N T IA L F A IL U R E M O D E A N D E F F E C T S A N A L Y S IS
Im u n n o a s s a y D e v ic e S u b s y s te m C o m p o n e n t _ S u p p o r t_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ R e s p o n s ib ility X P ro c e s s C o re T e a m :
_______________________________________________________________________________________________
FMEA Num ber Page of P re p a re d B y F M E A D a te (O rig .)
(r e v .)
D e v ic e / F u n c tio n
P o te n tia l F a ilu r e Mode
P o te n tia l E ffe c t(s ) o f F a ilu r e
P o te n tia l C a u s e (s ) o f F a ilu r e
C u rre n t C o n tr o ls
R P N
R e c o m m e n d e d R e s p o n s ib ilit A c tio n R e s u lts y A c tio n (s ) a n d T a rg e t A c tio n s R C o m p le te D a te T a k e n S O D P N
S u p p o rt S y s te m
H o ld s A n tib o d y
-
S e p a r a te s A n a ly te
F a ils to S e p a r a te a n a ly te
N o An sw er E rro n e o u s R e s u lt F a ls e H i F a ls e L o
W ro n g M a te r ia l
60
7 9
3 3
5 5
105 135
D e v e lo p s C o lo r
38
Process FMEA
Identifies potential product-related process failure modes Assesses the potential customer effects of the failures Identifies the potential internal and external manufacturing or assembly process causes Identifies process variables on which to focus controls for
reducing occurrence or increasing detection of the failure conditions
Sources of Process Defects

Omitted processing Processing errors Errors setting up work pieces Missing parts Wrong parts Adjustment error Processing wrong work piece Mis-operation Equipment not set up properly Tools and fixtures improperly prepared
40
FMEA Summary
Powerful tool for summarizing:
Important modes of failure Factors causing these failures Effects of these failures Risk prioritization Identifying plan to control and monitor Cataloging risk reduction activities
41
HAZOP
Haz ard and Op erability Study
Bottom up analysis Deviations from design intentions Systematic brainstorming based on guide words
42
HAZOP
Guide Words
No/Not More Less As well as Other than
43
Hazop Model
Design Statement Activity Transfer Material Powder Destination Hopper
44
HAZOP
Transfer
No Valve closed Line blocked Pump broken Pump fast
Powder
Tank empty
Hopper
Valve closed Hopper full
More
Other than
Larger tank Inaccurate gage Liquid Wrong powder
45
HAZOP Plan
Guide Deviation Causes Risk NO Powder flow Valve Low closed Line Med blocked Pump Med broken Action Interlock Operator training PM Who
46
HACCP
Risk Management System
Biological Hazards Chemical Hazards Physical Hazards
Requires
Prerequisite Quality System Program Traditionally GMPs
HACCP Steps
1. Conduct hazard analysis and identify 2. 3. 4. 5. 6. 7.
preventive measures Identify Critical Control Points. Establish critical limits Monitor each critical control point Establish corrective action to be taken when deviation occurs Establish verification procedures Establish record-keeping system
AACC Expert Access Live on Line 48
HACCP Decision Tree

P re v e n tiv e M e a s u re s E x is t M o d ify S te p Yes No Yes
S te p E lim in a te or R educe
No Excess C o n ta m in a tio n P o s s ib le No
Is c o n tr o l N e c e s s a ry fo r S a fe ty
Yes Yes Subsequent S te p s e lim in a te or R educe Yes No
No C r itic a l C o n tro l P o in t S to p Not a CCP
49
HACCP Worksheet
Firm Name: Firm Address: Product Description: Method of Storage and Distribution Intended Use and Consumer
1 Material/ processin g step 2 Identify potential hazards introduced, controlled or enhanced at this step(1) 3 Are any potential safety hazards significant? (Y/N) 4 Justify your decisions for column 3. 5 What preventative measures can be applied to prevent the significant hazards? 6 Is this step a critical control point? (Y/N)
Biological Chemical Physical

2004 Menson & Associates, Inc AACC Expert Access Live on Line
50
HACCP Plan
Firm Name: Firm Address Product Description: Method of Storage and Distribution: Intended use and Consumer: (1) Critical Control Point (2) Significan t Hazards (3) Critical Limits for each Action (4) What (8) Corrective Actions (7) Who (9) Record s (10) Verification
Monitoring
(5) How
(6) Frequency
51
Contact Information
Robert C. Menson, PhD Menson & Associates, Inc
Quality Systems and Compliance
For FDA Regulated Industries
PO Box 480 Newport, RI 02840 Phone: 1.401.847.6320 Email: r.menson@alumni.duke.edu


Risk Assessment Tools For IVD Assays

Uploaded by

Document Information

Original Description:

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Risk Assessment Tools For IVD Assays

Uploaded by

Copyright:

Available Formats

Risk Assessment Tools for Identifying Hazards and Evaluating Risks Associated with IVD Assays

Robert C. Menson, PhD

What Risks Must Be Managed?

Elements of the Risk Management Process

Risk Assessment Tools

Applications of Risk Analysis

2004 Menson & Associates, Inc

AACC Expert Access Live on Line

Concept & Feasibility

Scale-Up & Transfer

Design Control Risk Assessment

Productions Methods Risk Management Report

Preliminary Hazard Analysis

Risk Management Plan

Detailed Analysis (FMEA, FTA, HACCP, HAZOP)

2004 Menson & Associates, Inc

AACC Expert Access Live on Line

Key Concepts of Risk

The consequences of that loss;

The perception of the loss;

Risk Management Terms

2004 Menson & Associates, Inc

2004 Menson & Associates, Inc

ISO/IEC Guide 51:1999, definition 3.5

ISO/IEC Guide 51:1999, definition 3.2

ISO/IEC Guide 51:1999, definition 3.9

ISO/IEC Guide 51:1999, definition 3.7

2004 Menson & Associates, Inc

Events MDRs Accidents Etc

Brainstorming PHA FTA FMEA

2004 Menson & Associates, Inc

AACC Expert Access Live on Line

Severity of Consequence Codes

Potential serious injury. Not permanent

2004 Menson & Associates, Inc

AACC Expert Access Live on Line

2004 Menson & Associates, Inc

AACC Expert Access Live on Line

Estimation of Risk Codes

ALARP=As Low As Reasonably Possible

2004 Menson & Associates, Inc

AACC Expert Access Live on Line

ALARP Broadly Acceptable Region

Maximum Tolerable Risk

Increasing Severity of Harm

2004 Menson & Associates, Inc

AACC Expert Access Live on Line

FMEA vs. FTA

2004 Menson & Associates, Inc

AACC Expert Access Live on Line

FTA Basic Symbols

2004 Menson & Associates, Inc

FTA During Design

Competing Design Criteria

A S P E C T R A T IO TO SM ALL BASE FLATNESS B A S E S IZ E TO O SM ALL R IM T O F IL L IN S U F F C IE N T

2004 Menson & Associates, Inc

AACC Expert Access Live on Line

FTA Lab Failure

2004 Menson & Associates, Inc

AACC Expert Access Live on Line

FTA During Reliability

2004 Menson & Associates, Inc