1

PET/CT Performance Evaluation

Techniques and
Quality Assurance
Brad Kemp, PhD Brad Kemp, PhD
Department of Radiology Department of Radiology
Mayo Clinic Mayo Clinic
WE-SAMS-L100J-2
Outline
PerformanceEvaluationTechniques PerformanceEvaluationTechniques
NEMANU2 NEMANU2 2007PerformanceStandards 2007PerformanceStandards
QualityAssurance QualityAssurance
SystemCorrections SystemCorrections
QuantitativeAnalysisofBlankScans QuantitativeAnalysisofBlankScans
QuantitativePET QuantitativePET
DailyQA DailyQA
NEMA NU 2-2007
PETPerformanceStandard PETPerformanceStandard
NationalElectricalManufacturers NationalElectricalManufacturers
Association. Association.
NEMAStandardsPublicationNU2 NEMAStandardsPublicationNU2 2007: 2007:
PerformanceMeasurementsofPositronEmission PerformanceMeasurementsofPositronEmission
Tomographs Tomographs
http://www.nema.org/stds/nu2.cfm
NEMA NU 2-2007 and Lutetium-176
LSO(andLYSO)isinherentlyradioactive LSO(andLYSO)isinherentlyradioactive
BackgroundradiationgivesrisetoRandoms,someTrues BackgroundradiationgivesrisetoRandoms,someTrues
ImplicationsforNEMA: ImplicationsforNEMA:
cannotobtain cannotobtainRandoms:Trues Randoms:Trues ratio<1% ratio<1%
Forcountrateandsensitivitymeasurements Forcountrateandsensitivitymeasurements measurerandoms measurerandoms
ratesdirectly ratesdirectly
WatsonCC,etal. WatsonCC,etal.J JNucl Nucl Med Med 2004;45:822 2004;45:822 826. 826.
ErdiYE,etal. ErdiYE,etal.J JNucl Nucl Med Med 2004;45:813 2004;45:813 821. 821.
Kemp,etal. Kemp,etal.J JNucl Nucl Med Med 2006;47:1960 2006;47:1960 1967. 1967.
NU2 NU2 2007addressesintrinsicradioactivityindetectors 2007addressesintrinsicradioactivityindetectors
NEMA NU 2-2007
PerformanceMeasures: PerformanceMeasures:
SpatialResolution(Transaxial,axial) SpatialResolution(Transaxial,axial)
Sensitivity Sensitivity
* *
ScatterFraction ScatterFraction
* *
CountRatePerformance CountRatePerformance
* *
CountRateCorrectionAccuracy CountRateCorrectionAccuracy
ImageQuality ImageQuality
Phantoms for NU 2-2007
NEMANU2 NEMANU2 2007 2007
Threephantoms: Threephantoms:
Scatterphantom Scatterphantom
203x700mmphantomwithactivity 203x700mmphantomwithactivity
inlinesource inlinesource
ImageQualityphantom ImageQualityphantom
Sensitivityphantom Sensitivityphantom
2
Spatial Resolution
Spatialresolutionofasystemrepresentsitsabilitytodisting Spatialresolutionofasystemrepresentsitsabilitytodistinguish uish
betweentwopointsofradioactivity betweentwopointsofradioactivity
F F 18pointsourcesinairat 18pointsourcesinairatsix six locations: locations:
(0,1),(0,10)and(10,0)cm (0,1),(0,10)and(10,0)cm
CenterofaxialFOVand CenterofaxialFOVand axialFOVfromcenter axialFOVfromcenter
Reconstruct:imagepixel< Reconstruct:imagepixel< expectedFWHM expectedFWHM
Profilewidth~2timesFWHM Profilewidth~2timesFWHM
Spatial Resolution
ReportaverageFWHMandFWTMinradial,tangentialand ReportaverageFWHMandFWTMinradial,tangentialand
axialdirections axialdirections
At1cmradius: At1cmradius:
Transverse Transverse(averagexandy;4numbers) (averagexandy;4numbers)
Axial Axial(2numbers) (2numbers)
At10cmradius: At10cmradius:
Transverseradial Transverseradial(4numbers) (4numbers)
Transversetangential Transversetangential(4numbers) (4numbers)
Axial Axial(4numbers) (4numbers)
Sensitivity
Sensitivityofascannerrepresentsitsabilitytodetect Sensitivityofascannerrepresentsitsabilitytodetect
annihilationradiation annihilationradiation
Rateoftruecoincidencecountsperunitradioactivity(express Rateoftruecoincidencecountsperunitradioactivity(expressed ed
incps/ incps/kBq kBq)inabsenceofattenuatingmedia )inabsenceofattenuatingmedia
Rationale Rationale:needmaterialaroundsourcetoensureannihilationof :needmaterialaroundsourcetoensureannihilationof
positrons,butthismaterialalsoattenuatestheannihilation positrons,butthismaterialalsoattenuatestheannihilation
photons photons
BasedontechniquebyBaileyDL,JonesT,etal. BasedontechniquebyBaileyDL,JonesT,etal.Eur Eur J JNucl Nucl Med Med
1991;18:374 1991;18:374 379. 379.
Sensitivity
Successivemeasurementswitha700mmlinesourcewithaknown Successivemeasurementswitha700mmlinesourcewithaknown
amountofF amountofF 18surroundedbynested,knownabsorbers 18surroundedbynested,knownabsorbers
Randomeventrate<5%truesrate;countlosses<1% Randomeventrate<5%truesrate;countlosses<1%
Forsystemswithintrinsicradioactivity,measureandsubtract Forsystemswithintrinsicradioactivity,measureandsubtract
randomsratefrompromptsratepriortocalculations randomsratefrompromptsratepriortocalculations
Sensitivity
Thecountratewithnoabsorberisextrapolatedfromthecountr Thecountratewithnoabsorberisextrapolatedfromthecountrate ate
measuredwiththesleeves measuredwiththesleeves
0.0E+00
5.0E+04
1.0E+05
1.5E+05
0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4
Sleeve Thickness (cm)
C
o
u
n
t
R
a
t
e
, R

(
c
p
s
)
R
0
Activity
R
y Sensitivit System
0
=
Sensitivity
0.00
0.01
0.02
0.03
0.04
0.05
0 10 20 30 40 50
Slice
S
e
n
s
itiv
ity
(
c
p
s
/k
B
q
)
R0
R10
Measureatradiallocationsof0and10cm Measureatradiallocationsof0and10cm
Reportsystemsensitivityandslicesensitivityprofile Reportsystemsensitivityandslicesensitivityprofile
0.00
0.10
0.20
0.30
0.40
0 10 20 30 40 50
Slice
S
e
n
s
itiv
ity
(
c
p
s
/k
B
q
)R0
R10
2D 3D
3
Scatter Fraction and Count Losses
ScatterFractionisameasureofthesystemsensitivitytoscatt ScatterFractionisameasureofthesystemsensitivitytoscatter er
Trues Scatter
Scatter
SF
+
=
Countrateperformancegivesanindicationofscannerperformanc Countrateperformancegivesanindicationofscannerperformance e
asafunctionofactivity asafunctionofactivity
CalculateNoiseEquivalentCountRate CalculateNoiseEquivalentCountRateR R
NEC NEC
:Figureofmeritrelating :Figureofmeritrelating
scannerperformanceto scannerperformancetosinogram sinogram SNRafterrandomsandscatter SNRafterrandomsandscatter
corrections corrections
Randoms Scatter Trues
Trues
NEC
R k R R
R
R
+ +
=
2
Scatter Fraction and Count Losses
Acquisition Acquisition
Use203mmdiameterpolyethylenecylinderoflength700mm, Use203mmdiameterpolyethylenecylinderoflength700mm,
withaF withaF 18linesourceofdiameter2.3mmthatis4.5mmoffaxis 18linesourceofdiameter2.3mmthatis4.5mmoffaxis
MeasuredwithhighinitialactivityofF MeasuredwithhighinitialactivityofF 18 18
3D:800 3D:800MBq MBq;2D:5 ;2D:5GBq GBq
Acquiredatauntiltrueeventlosses<1%(14to18 Acquiredatauntiltrueeventlosses<1%(14to18hrs hrs) )
Forsystemswithintrinsicradioactivitymeasurerandoms Forsystemswithintrinsicradioactivitymeasurerandoms
delayedeventorcalculatefromsinglesrate delayedeventorcalculatefromsinglesrate
Ifthereisnorandomsmeasurementacquireuntil IfthereisnorandomsmeasurementacquireuntilRandoms:Trues Randoms:Trues
ratio<1% ratio<1%
Scatter Fraction and Count Losses
Processing Processing
Twomethods: Twomethods:
1. 1. Analysiswithrandomsestimate Analysiswithrandomsestimate
Plotscatterfractionasafunctionofcountrate Plotscatterfractionasafunctionofcountrate
ReportscatterfractionatthepeakNECR ReportscatterfractionatthepeakNECR
2. 2. Alternativeanalysiswithnorandomsestimate: Alternativeanalysiswithnorandomsestimate:
Calculatescatterfractionfromthoseacquisitionswith Calculatescatterfractionfromthoseacquisitionswith
Randoms:Trues Randoms:Trues ratio<1%. ratio<1%.
Reportsinglescatterfractionvalue Reportsinglescatterfractionvalue
0
4
8
12
16
20
0 20 40 60 80 100
2D activity concentration (kBq/mL)
S
c
a
tt
e
r
F
r
a
c
t
io
n
(
%
)
0
10
20
30
40
0 5 10 15 20 25
3D activity concentration (kBq/mL)
S
c
a
t
te
r
F
r
a
c
tio
n

(
%
)
Scatter Fraction
2D 3D
ScatterFractionvsCountRate ScatterFractionvsCountRate
Scatter Fraction
0
5
10
15
20
25
30
35
40
0 10 20 30 40 50
Slice
S
c
a
tte
r F
r
a
c
tio
n
(%
)
0
5
10
15
20
0 10 20 30 40 50
Slice
S
c
a
tte
r F
ra
c
tio
n
(
%
)
2D 3D
ScatterFractionvsSlice ScatterFractionvsSlice
Count Rate Performance
CalculateNoiseEquivalentCountRate CalculateNoiseEquivalentCountRate
Randoms Scatter Trues
Trues
NEC
R k R R
R
R
+ +
=
2
Reportpeaktruecountrate,peakNECandeffectiveactivity Reportpeaktruecountrate,peakNECandeffectiveactivity
concentrationsatthesepeaks concentrationsatthesepeaks
ForNEMA,k=1(calculatedRandoms);R ForNEMA,k=1(calculatedRandoms);R
NEC NEC
(1R) (1R)
4
Count Rate Performance
Peak PeakR R
Trues Trues
:454kcps@30.8 :454kcps@30.8kBq kBq/ml /ml
PeakR PeakR
NEC NEC
(1R):118kcps@22 (1R):118kcps@22kBq kBq/ml /ml
PeakR PeakR
NEC NEC
(2R):80kcps@18 (2R):80kcps@18kBq kBq/ml /ml
0
40
80
120
160
0 10 20 30 40
Activity concentration (kBq/mL)
N
E
C
R
(
k
c
p
s
)
NECR 1R
NECR 2R
0
500
1,000
1,500
2,000
2,500
0 10 20 30 40
Activity concentration (kBq/mL)
C
o
u
n
t R
a
te
(k
c
p
s
)
Total
Trues
Delays
Scatter
Image Quality Measurement
Standardizedimagingsituationthatsimulatesaclinicalwhole Standardizedimagingsituationthatsimulatesaclinicalwhole
bodyimagingcondition bodyimagingcondition
Phantomconsistsofatorsophantomwithhotandcoldlesionsin Phantomconsistsofatorsophantomwithhotandcoldlesionsin
awarmbackground awarmbackground
Scatterphantomabuttedtoimagequalityphantom Scatterphantomabuttedtoimagequalityphantom
Image Quality Measurement
PhantomDescription PhantomDescription
Hotspheres:10,13,17,22mmid Hotspheres:10,13,17,22mmid
Coldspheres:28,37mmid Coldspheres:28,37mmid
Lunginsert Lunginsert
Activityinhotspheres8and4timesthatofbackground Activityinhotspheres8and4timesthatofbackground
Activityinbackground5.3 Activityinbackground5.3kBq kBq/ml /ml
Activityinscatterphantom(116 Activityinscatterphantom(116MBq MBq) )
17mmspherealonghorizontalaxisofphantom 17mmspherealonghorizontalaxisofphantom
AbutscatterphantomtoIQphantom AbutscatterphantomtoIQphantom
Image Quality Measurement
Acquisition Acquisition
Centerofspheresisatthemiddlesliceofthescanner Centerofspheresisatthemiddlesliceofthescanner
Simulatedacquisition100cmin60min Simulatedacquisition100cmin60min
Repeatacquisitionthreetimes Repeatacquisitionthreetimes
Reconstructusingclinicalprotocol Reconstructusingclinicalprotocol
min 7
cm 100
min 60
= z T
acq
Image Quality Measurement
Reportimagecontrastand ReportimagecontrastandSNRs SNRs forhotandcoldlesions, forhotandcoldlesions,
residualerrorinlung,variabilityinbackground residualerrorinlung,variabilityinbackground
Visualinspectionforartifacts Visualinspectionforartifacts
Quality Assurance
QualityAssuranceTestRequirements QualityAssuranceTestRequirements
Well Well definedregimenofmeasurements definedregimenofmeasurements
Quickandeasytoconduct Quickandeasytoconduct
Sensitivetomodesoffailureofthescanner Sensitivetomodesoffailureofthescanner
Preferablyquantitative,notqualitative Preferablyquantitative,notqualitative
QualityAssurance: QualityAssurance:
Verifyscannerisoperatingproperly Verifyscannerisoperatingproperly
Identifyproblemspriortoscanningpatients Identifyproblemspriortoscanningpatients
Acquisitionofcorrectionstocompensateforknownimperfections Acquisitionofcorrectionstocompensateforknownimperfections
indatameasurement indatameasurement
5
Quality Assurance
VerifythatSystemCorrectionsarestillvalid VerifythatSystemCorrectionsarestillvalid
Normalization Normalization
Arevariationsindetectorefficienciesbeingcorrectedfor? Arevariationsindetectorefficienciesbeingcorrectedfor?
Scannercalibration Scannercalibration
aretheimagesmeasuringactivityconcentrationaccurately? aretheimagesmeasuringactivityconcentrationaccurately?
System Correction: Normalization
Purpose: Purpose: Correctsforthevariationsinefficiencyinlinesofresponse Correctsforthevariationsinefficiencyinlinesofresponse
(LOR)ineachsliceofthesinogram (LOR)ineachsliceofthesinogram
Auniformitycorrection Auniformitycorrection
For2D: For2D: DirectMeasurementusingalowactivitysource DirectMeasurementusingalowactivitysource
For3D: For3D: IndirectMeasurementusingauniformphantom IndirectMeasurementusingauniformphantom
Acquiredquarterlyoraftersystemmaintenance Acquiredquarterlyoraftersystemmaintenance
Sinogramofa2DRodSource
System Correction: Normalization
Before
Normalization
SinogramofaUniformCylinder
After
Normalization
QA Data: Blank Scan
Onstand Onstand alonePETsystems,usedwithtransmissionsinograms alonePETsystems,usedwithtransmissionsinograms
tocreateattenuationcorrectionfactors tocreateattenuationcorrectionfactors
Acquireddaily Acquireddaily goodsourceofQAdata goodsourceofQAdata
Emissionscanofuniformcylinder Emissionscanofuniformcylinder
PET/CTscanners PET/CTscanners normalizationsource normalizationsource
Badblock Goodblank
Sinogramofa
UniformCylinder
TransmissionSinograms
Quantitative Analysis of Blank Scans
Thequantitativeanalysisofblankscansisanimportantaspect Thequantitativeanalysisofblankscansisanimportantaspectof of
theQAofaPETscanner theQAofaPETscanner
Quantitativeanalysisofblankscans: Quantitativeanalysisofblankscans:
validatesdetectorcharacterization validatesdetectorcharacterization
providesinformationoncrystal,blockormodule(bucket) providesinformationoncrystal,blockormodule(bucket)
efficiencieswithrespecttosystemaverage efficiencieswithrespecttosystemaverage
comparetoreference comparetoreference monitorssystemstability monitorssystemstability
ensuresvalidityofnormalization ensuresvalidityofnormalization
Has the systemdrifted?
Is the normalization valid?
Quantitative Analysis of Blank Scans
Time Line
Normalization Normalization
Reference Blank
Has the systemdrifted?
Is the normalization valid?
DailyBlank
6
Quantitative Analysis of Blank Scans
DailyQA DailyQA analysiscalculatesaglobalandslicenormalizedmeansquare analysiscalculatesaglobalandslicenormalizedmeansquare
error(NMSE)byusingareferencescan;thereferencescanisde error(NMSE)byusingareferencescan;thereferencescanisdefinedasthe finedasthe
blankscanacquiredimmediatelyafternormalization blankscanacquiredimmediatelyafternormalization
Normalized Mean Square Error
Reference Blank DailyBlank
( )
( )
2
2

=
i
REF
i
i
DAY
i
REF
i
Counts
Counts Counts
NMSE
ComputeglobalandsliceNMSE
Pixeltopixelcomparison Pixeltopixelcomparison
NMSE Calculation
NMSE: NMSE:
calculatedfromcounts/pixelinblankscansinograms calculatedfromcounts/pixelinblankscansinograms
mustreflectchangesinscannerperformance, mustreflectchangesinscannerperformance,not not changesin changesin
thesource thesource
Therefore,acquireallblankscansforthesametotalcounts Therefore,acquireallblankscansforthesametotalcounts
Example: NMSE and Block Bias
0.000
0.002
0.004
0.006
0.008
0.010
10-Dec-02 29-J an-03 20-Mar-03 9-May-03 28-J un-03 17-Aug-03
Date
N
o
r
m
a
liz
e
d

M
e
a
n

S
q
u
a
r
e

E
r
r
o
r
-8
-6
-4
-2
0
2
4
6
B
lo
c
k

B
ia
s

(
%
)
Global NMSE
Slice NMSE
Block Bias
QAmustbesensitivetomodesoffailureofthescanner QAmustbesensitivetomodesoffailureofthescanner
NMSE:Errorwithrespecttobaselinescan NMSE:Errorwithrespecttobaselinescan
Quantitative Analysis of Blank Scans
Twoadjacentdirectslices( Twoadjacentdirectslices(ie ie adjacentrings,rowsinblock) adjacentrings,rowsinblock)
System Calibration
Purpose: Purpose:Definesrelationshipbetweenthemeasuredcount Definesrelationshipbetweenthemeasuredcount
ratepervolumeandthetrueactivityconcentration ratepervolumeandthetrueactivityconcentration
Aphantomofknownactivityconcentrationisimagedand Aphantomofknownactivityconcentrationisimagedand
reconstructedwithuniformattenuationcorrection reconstructedwithuniformattenuationcorrection
Averagecountsinaslicearecalculated Averagecountsinaslicearecalculated
Fromthisasystemcalibrationiscomputed Fromthisasystemcalibrationiscomputed
Convertimagecounts/pixeltoactivity/volume Convertimagecounts/pixeltoactivity/volume
Acquiredmonthlyoraftersystemmaintenance Acquiredmonthlyoraftersystemmaintenance
7
Quantitative PET
Example Example:FDGPETstudiesformonitoringtumorresponse :FDGPETstudiesformonitoringtumorresponse
Assumethattheleveloftumoruptakeisproportionalto Assumethattheleveloftumoruptakeisproportionalto
tumor tumor activity activity
Quantitativeanalysis Quantitativeanalysis
Standardizeduptakevalue Standardizeduptakevalue
Patlak Patlak analysis analysis
Compartmentalmodel Compartmentalmodel
Requires Requires
Scannertobecross Scannertobecross calibratedwithdosecalibrator calibratedwithdosecalibrator
Scanner(anddosecalibrator)tobestable Scanner(anddosecalibrator)tobestable
Therefore,acquiresystemcalibrationmonthly Therefore,acquiresystemcalibrationmonthly
Quantitative PET: SUV
FormulticenterstudiesrelyingonquantitativePET,scanners FormulticenterstudiesrelyingonquantitativePET,scanners
mustbecertifiedbeforeparticipationinthetrial mustbecertifiedbeforeparticipationinthetrial
ACRINscannercertificationprocedure: ACRINscannercertificationprocedure:
ImageauniformphantomofF ImageauniformphantomofF 18usingtypicalclinical 18usingtypicalclinical
protocol protocol
Reconstruct,applyingallcorrections Reconstruct,applyingallcorrections
DisplayinSUVunits,obtainaverageincentralROIforeach DisplayinSUVunits,obtainaverageincentralROIforeach
slice slice
Shouldget0.9<SUV<1.1 Shouldget0.9<SUV<1.1
Daily Quality Assurance
CantheScannerbeUsedToday? CantheScannerbeUsedToday?
PotentialProblems PotentialProblems
Systemstability,drifts Systemstability,drifts
Detectormodule/PMT/preampfailure Detectormodule/PMT/preampfailure
Loosecables,connectors Loosecables,connectors
Inoperablegantrymotors,sourceloader Inoperablegantrymotors,sourceloader
DailyQARegimen DailyQARegimen
AutomaticPMTgainadjustment AutomaticPMTgainadjustment
Blankscan Blankscan
QAwilldetectbutnotpreventtheseproblems QAwilldetectbutnotpreventtheseproblems
Daily QA: Loose Cable
LooseCable Better
QA Schedule
DetectorandElectronics DetectorandElectronics
Characterization Characterization Frequency Frequency
Crystalmap Crystalmap Quarterly Quarterly
PMTgain PMTgain Daily Daily
Coincidencetiming Coincidencetiming Weekly(Daily) Weekly(Daily)
SystemCorrections SystemCorrections
Normalization Normalization Quarterly Quarterly
Scannercalibration Scannercalibration Monthly(Weekly) Monthly(Weekly)
BlankScan BlankScan Daily Daily