A brief overview of ROC Analysis:

ROC analysis has been mainly used to assess diagnostic checks.

The ROC analysis starts with the observer assigning exactly two classifications to
a case after observing one stimulus associated with the case. The two
classification states are whether there is presence of a particular disease or not
and normal and abnormal.
Bases on the information from imaging exam the observers task is to determine
the correct disease classification. Then the observers classification is compared
to the true classification of the reference variable. Subsequently, sensitivity and
specificity measures are constructed.
Sensitivity is the proportion of correctly specified cases among those who are
truly positive.
Specificity is the proportion of correctly specified cases among those who are
truly negative.
In real world classification is associated with uncertainty therefore sensitivity and
specificity can never be 100. The trade-off between sensitivity and specificity
depends on the threshold set by the observer. Low threshold shows tendency to
over-call high sensitivity relatively to low specificity. On the other hand higher
threshold shows the tendency to under-call which refers to low sensitivity to
relatively low specificity.The horizontally flipped plot of sensitivity over specificity
constitutes the so called ROC curves graph. The area under the ROC curve (AUC)
is the global indicator of the diagnostic performance of the test and represents
the probability of the observer correctly identifying the positive case with a
randomly chosen pair of cases where one is positive and the other one is
negative. AUC also represents the average over entire range of possible
sensitivities.
The most important property of ROC curves is independence from the threshold
set by the observer when choosing. It is a more accurate measure as it covers all
possible interpretation thresholds.
Parametric methods are used when the statistical distribution of test values in
diseased and non-diseased is known. Binormal distribution is commonly used for
this purpose. This is applicable when both diseased and non-diseased test values
follow normal distribution. If data are actually binormal or a transformation such
as log, square or Box- Cox makes the data binormal then the relevant
parameters can be easily estimated by the means and variances of test values in
diseased and non-diseased subjects.
Figure 1

Evaluation of optimal BMI cut-off point.

Three criteria are used to find optimal threshold point from ROC curve. First two
methods give equal weight to sensitivity and specificity and impose no ethical,
cost, and no prevalence constraints. The third criterion considers cost which
mainly includes financial cost for correct and false diagnosis, cost of discomfort
to person caused by treatment, and cost of further investigation when needed.
This method is rarely used in medical literature because it is difficult to estimate
the respective costs and prevalence is often difficult to assess. These three
criteria are known as the points on curve closest to the (0, 1)(shortest distance
(C) , Youden index (J), and minimize cost criterion, respectively.
Based on model 1, the first criterion the optimal cut-off points of the
classification variable in the ROC analysis can be calculated from the shortest
distance between the ROC curve and the left hand side of the ROC curve where
sensitivity and specificity equal 1. If sn and sp denote sensitivity and specificity,
respectively, the distance between the point (0, 1) and any point on the ROC
curve is d = [(1 sensitivity) 2+ (1 specificity)2.To obtain the SN= Sensitivity
(True positive rate), SP Specificity (False positive rate) optimal cut-off point to
discriminate the disease with non-disease subject, we need to calculate this
distance for each observed cut-off point, and locate the point where the distance
is minimum.
The second is Youden index that maximizes the vertical distance from line of
equality to the point B[x, y] as shown in the Figure 1 . The x-axis represents (1specificity) and y-axis represents sensitivity. In other words, the Youden index J is
the point on the ROC curve which is farthest from line of equality (diagonal line).
The main aim of Youden index is to maximize the difference between TPR
(sensitivity) and FPR (1 specificity) and little algebra yields J = max [sensitivity
(B)+specificity(B)-1]. The value of J for continuous test can be located by doing a
search of plausible values where sum of sensitivity and specificity can be
maximum. Youden index is more commonly used criterion because this index
reflects the intension to maximize the correct classification rate and is easy to
calculate. The index is defined for all points of an ROC curve, and the maximum

value of the index may be used as a criterion for selecting the optimum cut-off
point when a diagnostic test gives a numeric rather than a dichotomous result.