AUTOVERIFICATION:The HOW

Michael W. Fowler, Ph.D.

Distinguished Professor of Chemistry and Biology Oklahoma Christian University Oklahoma City, OK

Autoverification Rule And Retain-For-Review Process Implementation Brainstorming

Data Gathering Data Evaluation

IF PROBLEMS

Autoverification Rule and Retain-For-Review PreImplementation Testing

Autoverification Rule and Retain-ForReview Process Definitions

Autoverification Rule and Retain-For-Review Process Implementation

Periodic Autoverification Rule and Retain-For-Review Process Verification

Autoverification Rule and Retain-For-Review Process Monitoring

IF PROBLEMS

BRAINSTORMING
Involve Staff
Bench Technologists Lead Technologists/Technical Specialists Managers/Coordinators Scientific/Technical and Medical Directors IS&T

Make list of all necessary information specimen types, instrument error messages, etc.

Instrument Manual

DATA GATHERING

Specimen Parameters Instrument Flags Analytical Performance and Quality Control Information Reference Range Considerations Reference Change Values and Delta Checks Look-back Interval Clinical Practice Guidelines and Information Concurrent Test Information

Specimen Parameters
Specimen Type, Acceptability and Analysis Bias
Serum Plasma ( EDTA, heparin, other) Whole Blood

Specimen Stability

Instrument Parameters
Specimen Validity Flags, Indices Flags and/or Quantification
Hemolysis Lipemia Icterus Others

Performance Flags
Abnormal Absorbance
Range Limits Insufficient Sampling Clots

Analytical and Quality Control Information

Method Imprecision Method Interferences Method Bias Automated QC monitoring

Reference Range Considerations

Age-related Issues - ALKP Gender-related issues hormones, tumor markers Ethnic-related issues Collection Issues
AM vs PM - hormones Recumbant vs ambulatory albumin Concurrent therapy (e.g. IV, K+)

Inpatient vs outpatient

Clinical Practice Guidelines

Published
Thyroid (2002) Diabetes Mellitus (2002) Cardiac Markers (1999) Hepatic Injury (2000)

Interviews with client physicians

Delta Checks: How to Determine?

Technologist Experience Data Review Clinician Input Literature Reference Change Values

Delta Checks and Reference Change Values

Calculation of RCV (Significant Change) Necessary Data
Analytical Variation (Imprecision)
Local, historical CV near decision points

Biologic Variation
Literature Sources
Fraser CG, Biologic Variation: From Principles To Practice, AACC Press, 2001 Ricos C, et.el., Current databases on biologic variation: pros, cons, and progress. Scand J Clin Lab Invest 1999;59:491-500 www.westgard.com/guest17.htm

Experimental Data hard to obtain locally

Delta Checks and Reference Change Values Assumptions

Pre-analytical Variation is minimized and constant Analytical Variation is less than of Biological Variation

Calculation of Reference Change Values

RCV = 21/2 x Z x (CVA2 +CVI2)1/2
Z Probability or Degree of Significance
Z= 2.58 at 99% Probability (Highly Significant) Z = 1.96 at 95% Probability (Significant) *Other Z values available in statistics tables

CVA Analytical Variation CVI Biological Variation (within-subject)

Calculation of Reference Change Values

Significant Change RCVsig(95%)= 21/2 x 1.96 (CVA2 +CVI2)1/2 Highly Significant Change RCV high sig(99%) = 21/2 x 2.58 (CVA2 +CVI2)1/2

Reference Change Values and Delta Checks

Example #1 Cholesterol
CVA=2.0% CVI=6.0%

RCVsig(95%)= 21/2 x 1.96 (2.02 +6.02)1/2 = 17.2% RCV high sig(99%) = 21/2 x 2.58 (2.02 +6.02)1/2 = 22.6%
Recommended Delta Check Value At least 23 % from previous result
At 250mg/dl +58 mg/dl At 300 mg/dl +69 mg/dl

Reference Change Values and Delta Checks

Example #2 TSH
CVA = 5.5% at 0.36 mU/L CVI= 19.7% RCVsig(95%)= 21/2 x 1.96 (5.52 +19.72)1/2 = 57% RCV high sig(99%) = 21/2 x 2.58 (5.52 +19.72)1/2 = 75%
Recommended Delta Check Value At least 75 % change from previous result
At 0.3 uU/ml +0.23 uIU/ml At 5.0 uU/ml +3.75 uIU/ml NACB Thyroid Testing Guidelines +0.75 mIU/L

Other Uses of Reference Change Values

Setting Quality Specifications for Tests Enhance Clinical Information
Use RCV information to Flag ComputerReported Results as Significant or Highly Significant Change

Delta Checks and Look-Back Interval

Acute Care vs Chronic Care vs Preventive Care
Example K+
Hopitalized hours or days Health Maintenance Profile months

Example TSH
Interval between Physician visits?

Screening vs Monitoring
Example PSA
Screening annually or more Monitoring after detection (>4.0) 6 mo to 1 yr Monitoring after Surgery - ?3 to 6 months Monitoring after Radiation Therapy - ?

Concurrent Test Information

Comparison to same-order/requisition results
Na+ to Cl- or Anion Gap TSH to FT4(3) or T4(3) BUN to Creatinine ALT to AST, Bilirubin Total Protein to Albumin Hemolysis Index and Troponin Hemolysis Index and K+

Autoverification Rule And Retain-For-Review Process Implementation

Brainstorming Data Gathering

Data Evaluation

IF PROBLEMS

Autoverification Rule and Retain-For-Review PreImplementation Testing

Autoverification Rule and Retain-ForReview Process Definitions

Autoverification Rule and Retain-For-Review Process Implementation

Periodic Autoverification Rule and Retain-For-Review Process Verification

Autoverification Rule and Retain-For-Review Process Monitoring

IF PROBLEMS

DATA EVALUATION
Use acquired data and workgroup experience to formulate rule

Decide where to implement each part of the rule

Instrument System Middleware LIS HIS

Autoverification Rule And Retain-For-Review Process Implementation

Brainstorming Data Gathering Data Evaluation

IF PROBLEMS

Autoverification Rule and Retain-For-Review PreImplementation Testing

Autoverification Rule and Retain-ForReview Process Definitions

Autoverification Rule and Retain-For-Review Process Implementation

Periodic Autoverification Rule and Retain-For-Review Process Verification

Autoverification Rule and Retain-For-Review Process Monitoring

IF PROBLEMS

AUTOVERIFY RULE DEFINITION

Define Autoverify Rules in proper form
IF statements e.g. IF result > or < Ref Range
Evaluate True or False or Then

Multiple IF statements linked

IF result > or < Ref Range and IF Delta Value > X% Evaluate True or False or Then

RETAIN-FOR-REVIEW PROCESS
Clearly define what to do when Autoverification Rule Fails (Retain-for Review Process) Use Brainstorming Group to Formulate Retainfor-Review Process
Example: If K+ > 5.5-6.0 mEq/L check for hemolysis; if not hemolyzed report; if hemolyzed report with comment or redraw

Do while developing Autoverification Rule (Linked Process)

Autoverification Rule And Retain-For-Review Process Implementation

Brainstorming Data Gathering Data Evaluation

IF PROBLEMS

Autoverification Rule and Retain-For-Review PreImplementation Testing

Autoverification Rule and Retain-ForReview Process Definitions

Autoverification Rule and Retain-For-Review Process Implementation

Periodic Autoverification Rule and Retain-For-Review Process Verification

Autoverification Rule and Retain-For-Review Process Monitoring

IF PROBLEMS

AUTOVERIFY RULE and RETAIN-FOR-REVIEW PRE-IMPLEMENTATION TESTING

Use computer test system to test Autoverify Rule against database Use database downloaded to spreadsheet to test Autoverify Rule Use manual testing of recently processed data

Autoverification Rule And Retain-For-Review Process Implementation

Brainstorming Data Gathering Data Evaluation

IF PROBLEMS

Autoverification Rule and Retain-For-Review PreImplementation Testing

Autoverification Rule and Retain-ForReview Process Definitions

Autoverification Rule and Retain-For-Review Process Implementation

Periodic Autoverification Rule and Retain-For-Review Process Verification

Autoverification Rule and Retain-For-Review Process Monitoring

IF PROBLEMS

AUTOVERIFY RULE
and

RETAIN-FOR-REVIEW PROCESS REVISION

Make necessary changes in the Autoverify Rule and Retain-for-Review Processes based on results of pre-implementation testing Retest modified Rule and Retain-for-Review Process

Autoverification Rule And Retain-For-Review Process Implementation

Brainstorming Data Gathering Data Evaluation

IF PROBLEMS

Autoverification Rule and Retain-For-Review PreImplementation Testing

Autoverification Rule and Retain-ForReview Process Definitions

Autoverification Rule and Retain-For-Review Process Implementation

Periodic Autoverification Rule and Retain-For-Review Process Verification Autoverification Rule and Retain-For-Review Process Monitoring

IF PROBLEMS

AUTOVERIFY RULE
and

RETAIN-FOR-REVIEW PROCESS IMPLEMENTATION

Just Do It!

Autoverification Rule And Retain-For-Review Process Implementation

Brainstorming Data Gathering Data Evaluation

IF PROBLEMS

Autoverification Rule and Retain-For-Review PreImplementation Testing

Autoverification Rule and Retain-ForReview Process Definitions

Autoverification Rule and Retain-For-Review Process Implementation

Periodic Autoverification Rule and Retain-For-Review Process Verification

Autoverification Rule and Retain-For-Review Process Monitoring

IF PROBLEMS

AUTOVERIFY RULE and RETAIN-FOR-REVIEW PROCESS MONITORING

Have Back Out Plan to remove rule

Monitor intensely for 24 48 hours!!

Monitor carefully for 1 2 weeks!

Monitor for 1 month

Autoverification Rule And Retain-For-Review Process Implementation

Brainstorming Data Gathering Data Evaluation

IF PROBLEMS

Autoverification Rule and Retain-For-Review PreImplementation Testing

Autoverification Rule and Retain-ForReview Process Definitions

Autoverification Rule and Retain-For-Review Process Implementation

Periodic Autoverification Rule and Retain-For-Review Process Verification

Autoverification Rule and Retain-For-Review Process Monitoring

IF PROBLEMS

AUTOVERIFY RULE and RETAIN-FOR-REVIEW PROCESS REVISION

Make necessary changes in the Autoverify Rule and Retain-for-Review Processes based on results of monitoring Retest Implement Monitor

Autoverification Rule And Retain-For-Review Process Implementation

Brainstorming Data Gathering Data Evaluation

IF PROBLEMS

Autoverification Rule and Retain-For-Review PreImplementation Testing

Autoverification Rule and Retain-ForReview Process Definitions

Autoverification Rule and Retain-For-Review Process Implementation

Periodic Autoverification Rule and Retain-For-Review Process Verification

Autoverification Rule and Retain-For-Review Process Monitoring

IF PROBLEMS

PERIODIC AUTOVERIFY RULE and RETAIN-FOR-REVIEW PROCESS VERIFICATION

Include Autoverify Rule Description, Retain-for-Review Process and Periodic Verification Process in SOP Design Periodic Verification Process to meet regulatory requirements

Autoverification Rule And Retain-For-Review Process Implementation

Brainstorming Data Gathering Data Evaluation

IF PROBLEMS

Autoverification Rule and Retain-For-Review PreImplementation Testing

Autoverification Rule and Retain-ForReview Process Definitions

Autoverification Rule and Retain-For-Review Process Implementation

Periodic Autoverification Rule and Retain-For-Review Process Verification

Autoverification Rule and Retain-For-Review Process Monitoring

IF PROBLEMS