Detection Accuracy in Chest Radiography

JOEL E. GRAY,”2 KENNETH W. TAYLOR,’ AND BARRY B. HOBBS’

Downloaded from www.ajronline.org by 103.119.54.48 on 10/27/19 from IP address 103.119.54.48. Copyright ARRS. For personal use only; all rights reserved

The detection accuracy of the diagnostic radiologist is combinations (Alpha 4/RP, Hi Plus/RP, Alpha 4/XD, and
important in everyday medical decision making. However, Alpha 4/XM) and three focal spots (0.3, 1 .0, and 2.0 mm
little work has been done relating the detection accuracy of nominal sizes). Though rare earth screen-film systems
the radiologist to the quality of the image. This study, using a are not conventionally used for chest radiography, the
thorax and lung phantom, simulated tissue-equivalent 6.4 mm
potential for patient dose reduction is significant. How-
lesions, and a 183 cm source-to-image distance, shows that
ever, it must be assured that the detection accuracy of
the detection accuracy is not dependent on the focal spot size
(over a range of 0.3-2.0 mm). However, the false positive rate the radiologist is not impaired by the noisier, lower dose
increases when using small focal spots. In addition, the image’s.
detection accuracy decreases with increasing root-mean- The major variables studied were image quality (mod-
square (RMS) noise (a measure of the amount of quantum ulation transfer function, MTF) and noise (mottle) with
mottle in the image), while the false positive rate and intraob- differences in responses between radiologists and non-
server disagreement increase with increasing RMS noise. It is radiologists also being considered. Images of a thorax
also shown that the nonradiologist responds to changes in phantom with lungs and simulated spherical lesions
noise in exactly the same way as the radiologist. were produced. So that the radiologists would not be-
Introduction come familiar with the distributions, 14 lesion patterns
were used.
It is often assumed that any degradation in image quality,
in terms of noise or image fidelity (resolution or modula- Materials and Methods
tion transfer function [MTF]), will significantly decrease
Experimental Films
the detection accuracy of the radiologist. This is an
important assumption because of the added cost re- Films were produced using three focal spots (0.3, 1 .0, and
quired to produce imaging equipment with improved 2.0 mm) and four screen-film combinations (table 1). In all
instances the modulation transfer function (fig. 1) was deter-
resolution and because of the additional dose required
mined by means of edge-gradient analysis [8]. Only one MTF is
to produce low noise radiographs. Feddema and Botden
shown for Alpha 4 (fig. 1A) since the MTF for all three combina-
[1] concluded that the radiologist can detect the perti-
tions using that screen were identical. The effect of phase was
nent pathology on images of extremely low resolution disregarded since it was found to be negligible for the focal
but, given the choice, would normally select the most spots used [9].
aesthetically pleasing high resolution image. The Alpha 4/RP system is a mismatched system because the
Most studies associating image quality with diagnostic film is not spectrally sensitive to the green emission of the rare
accuracy have had serious limitations in the types of earth phosphors. It was used in the study so that three systems
simulated radiographs used . Several authors [2-5] lim- with different noise levels and the same MTF could be studied.
ited their studies to relatively simple stimuli, such as The radiographs for this study (fig. 2) were produced on 28 x
uniformly exposed radiographs containing circular Ic- 35.5 cm film using a thorax phantom (3M Co.) consisting of
human bone encased in Plexiglas. The phantom lungs were dog
sions. Kundel and Revesz [6] studied this problem by
lungs inflated with formalin fumes while the vascular system
superimposing simulated lesions on conventional radio- was perfused with latex. The dried, preserved lungs simulated
graphs using a video system. This approach is limited by the vasculature of the human lung. No attempt was made to
the restricted response of video systems. More recently, simulate the mediastinal, diaphragmatic, and abdominal re-
Brogdon et al. [7] used radiographs produced by super- gions.
imposing normal radiographs and radiographs of simu- The lesions consisted of Lucite spheres 6.4 mm in diameter
lated pathology. However, the image quality of the dupli- (fig. 3). Several radiologists were shown chest radiographs
cate of the superimposed films was not established. containing 6.4 and 4.8 mm lesions in identical locations. They
Studies using clinical radiographs are usually not con- detected only one or two of the nine smaller lesions, whereas
clusive due to variations in the patient and the position they detected six to eight of the larger lesions. Consequently,
the lesions of larger size were used.
of the lesion. In such studies it is difficult to determine if
All radiographs were produced with the geometry shown (fig.
changes in detectability are due to differences in image
4) using an anteroposterior projection. The phantom was posi-
quality or to lesion location. tioned identically for each radiograph, assuring that the lesions
In an attempt to overcome these problems and to and superimposed structures were in the same positions on the
determine the effect of image quality on detection accu- radiographs. The radiographs were made at 90 kVp using
racy, we carried out a study using four screen-film stationary grids (10:1 ratio, 40 lines/cm with a 183 cm source-

Received November 9, 1977; accepted after revision March 16, 1978.

This work was supported by funds from the James Picker Foundation and the Edward Christie Stevens Foundation.
IRadiological Research Laboratories, University of Toronto, Toronto, Ontario, Canada.
S Present address: Diagnostic Radiology, Mayo Clinic and Mayo Foundation, Rochester, Minnesota 55901 . Address reprint requests to J. E. Gray.

Am J Roentgnol 131 :247-253, August 1978 247 0361 -803X/78/08-0247 $00.00

0 1978 American Roentgen Ray Society
 248 GRAY ET AL.

TABLE 1
Characteristics of Screen-Film Combinations

Root-
Rela-
Mean-
Screen-Film Screen tive
Film Type square
Combination Type Expo Noise
sure (x 10’)
Downloaded from www.ajronline.org by 103.119.54.48 on 10/27/19 from IP address 103.119.54.48. Copyright ARRS. For personal use only; all rights reserved

DuPont Hi PIus/
Kodak RP. ‘CaWO4 Blue-sensitive 0.90 0.56
3M Alpha 4:
With Kodak
RP Rare earth Blue-sensitive 1 .65 0.57
With 3M XD. Rare earth Green-sensitive 1 .00 0.61
With 3M XM. Rare earth Green-sensitive 0.30 0.73

A B
1.0

0.8

LI. 0.6
I-

0.4

0.2 High
Plus

-i I I I I I
F.,j. 2.-Radiograph of phantom with nine lesions. Obvious lesions,
0 2 4 6 8 10 0 2 4 6 8 10
deleted in analysis, are indicated by circles; remaining seven lesions are
Frequency Ic/mm) indicated by arrows.

Fig. 1.-Screen-film and focal spot modulation transfer function

(MTF). A, MTF for screen-film system as function of frequency. B, MTF
for three focal spots as function of frequency. In order to study the effect of focal spot degradation, an
additional eight lesion patterns (P7-P14) were used with each
of the three focal spots, two patterns for each of the four
to-image distance). Though higher kVp techniques are conven-
screen-film combinations (Alpha 4/RP; P7 and PB; Hi Plus/RP,
tionally used, it was necessary to limit the study to 90 kVp due
Pg and PlO; Alpha 4/XD, P11 and P12; and Alpha 4/XM, P13
to generator timing-circuit limitations. That is, the high speed
and P14). Only one subject (radiologist no. 8) mentioned that
rare earth screen-film combinations, when used at higher kilo-
voltages, require exposure times much shorter than many older
there seemed to be one or two patterns that were duplicated ,so
it seems that 14 sets of patterns were sufficient for the study.
generators are capable of accurately and consistently produc-
Not all of the radiologists read all of the 72 films in each session.
ing.
A Kodak M6AN X-Omat, using DuPont Cronex Multi-Process Film Reading Conditions
chemistry at 32#{176}C,
was controlled sensitometrically throughout
the experiment. A medium density of 1 .0 above the base-plus- The participants in the study were shown the films in a quiet
room with subdued light and with no interruptions. Each heard
fog (B + F) level was maintained to better than ±0.10 in optical
the same tape-recorded instructions (see Appendix). Several of
density, while the density difference between two points about
0.25 and 2.0 above B + F was maintained to better than ± 0.10 the participants viewed the radiographs multiple times with 2
in optical density. The density in the clear, circled region of the days to several weeks between sessions. The participants in-
radiograph (fig. 2) was maintained at 1 .0 ± 0.05 (1 SD) above cluded radiologists, residents, and nonradiologists (table 2). All
the B + F level of the film. This density was selected on the participants had visual acuity of 20/20 at 30 and 100 cm. One
radiologist (no. 1) and three nonradiologists (A, B, and C) knew
basis of the radiologists’ choice of a “diagnostic film” from
that a limited number of lesion patterns were used and that
radiographs made at various density levels.
All lesions were placed over the lung areas and were located there were nine lesions in each radiograph.
on the anterior surface of the phantom (with an anteroposterior A conventional viewbox, masked to a film size just under 28 x
projection). Each radiograph contained nine lesions; three were 35.5 cm was used . The viewbox brightness was 9 x lOl apostilbs
(Im/m2). The room was dimly illuminated at 100 lx (lm/m2). The
located in the intercostal spaces, three on the ribs, and three on
participants were asked to mark the correct lesion locations on
the edge of the ribs so that about half of the lesion was on the
rib and half was in the intercostal space. an 11 x 14 cm reproduction of the radiograph. The viewing
In order to study the effect of noise, six lesion patterns (P1- distances and times were measured by means of a television
P6) were used with each of the four screen-film combinations, tape recorder. However, no correlation could be found between
two patterns for each of the three focal spots (0.3 mm, P1 and these parameters and the MTF or noise of the radiographs.
P2; 1 .0, P3 and P4; 2.0, P5 and P6). The determination of the
Analysis of Data
inconsistency of the participants in reading identical films was
possible, since a second radiograph was made for each of the Several lesions were very obvious and were deleted from
24 screen-film-focal spot combinations, making a total of 48 further analysis, since changes in image quality would not affect
films. the detection accuracy for such lesions. This merely reduced
 DETECTION ACCURACY IN CHEST RADIOGRAPHY 249

Fig. 3.-Array of Lucite spheres:

9.5, 7.9, 6.4, 4.8, 4.0, 3.2, and 2.4
Downloaded from www.ajronline.org by 103.119.54.48 on 10/27/19 from IP address 103.119.54.48. Copyright ARRS. For personal use only; all rights reserved

mm in diameter. Lucite spheres cho-

sen to simulate lesions were 6.4 mm.
A, Kodak type M film without inten-
sifying screens and with no scatter
or focal spot degradation. B, Alpha
4-xM screen-film combination with
sufficient scatter to reduce lesion
contrast to level similar to that en-
countered in phantom images and
with no focal spot degradation.

TABLE 2
Summary of Subjects

5ubject 5ubject
ldentifica- Background ldentifica- Background
lion lion

1 Radiologist 8 . . Radiologist
2 Radiologist 9 Resident (fourth year)
3 Resident (third year) A Secretary
4 Radiologist B Graduate student
5 Radiologist C Medical physicist
6 Resident (second year) D Radiologic
7 Resident (first year) technologist

for deletion from the focal spot series were located correctly in
eight or nine of a possible nine responses and were clearly
visible on the films made with the 2.0 mm focal spot. Of the 126
lesions, 50 were deleted; of these 50, seven were lesions on
bone, 25 were lesions oft bone, and 18 were lesions on bone
edges.
To study the inconsistency, or intraobserver disagreement, of
a given observer, two identical radiographs were made with
each screen-film-focal spot combination. (All lesions, including
those deleted for the detection accuracy analysis, were included
in the intraobserver disagreement analysis). For each lesion that
the participant saw twice in the same session on films made with
the same focal spot and screen-film combination, there were
three possible responses: (1) the subject did not locate the
lesion correctly either time; (2) the subject located the lesion
Fig. 4.-Geometry for x-ray imaging system. correctly both times; and (3) the subject located the lesion
correctly only once. The ratio of the number of the responses
locating the lesion correctly only once to the total number of
lesion pairs is the intraobserver disagreement. The relative
the overall detection accuracy. The detection accuracy was intraobserver disagreement is then found by dividing the mdi-
then defined as: vidual’s intraobserver disagreement for a specific imaging con-
dition by his average disagreement, thereby reducing the varia-
A_C -O (1) tion due to differences between sessions and observers.
-

Much of the analysis used data from the three films in

where N is the number of lesions, 0 is the number of obvious combination with Alpha 4 screens. This assured that the
lesions, and C is the number of correct responses. The lesions changes noted were due to changes in the noise only and not to
deleted from the screen-film series were those that were cor- differences in the MTF of the intensifying screen.
rectly identified in 22-24 of 24 possible responses and were A count of the total number of false positive responses was
clearly visible in the radiograph made with the Alpha 4/XM made, and the average number per reading session was deter-
combination (the noisiest image). Likewise, the lesions selected mined.
 250 GRAY ET AL.

Results
0.8 3 3
3
Radiologists as Group
0.7
An attempt was made to determine the effect of 8
Fig. 5.-Detection accuracy
changes in image quality on the detection accuracy of
as function of nominal focal spot <
the radiologists using the responses from one session size for radiologists 1-9 (table 2). 2 99
for each of nine radiologists (fig. 5). Though trends may Averages for nine radiologists
Downloaded from www.ajronline.org by 103.119.54.48 on 10/27/19 from IP address 103.119.54.48. Copyright ARRS. For personal use only; all rights reserved

areindicatedbydots. 0.4 6 7
be apparent, it is not possible to determine from figure 5
if the changes in detection accuracy were significant, 4
0.3
due to the wide spread in the overall accuracies of the
0.3 1.0 2.0
individual radiologists. The wide range of individual Focal Spot Sin 1mm)
accuracies has not been considered in many previous
studies. It is clear that changes in detection accuracy
must be considered for each radiologist in order to
when reading films with a large amount of noise (fig.
determine whether some improve and some get worse
7C).
with changes in image quality.
Nonradiologists
Radiologists as Individuals
Four nonradiologists viewed the radiographs three
Five radiologists were asked to read the films several
times each and followed the same procedures as the
times. Only the change in the criterion level of the
radiologists (fig. 8). The nonradiologists responded to
individual radiologist and a possible learning effect
changes in image quality in a manner similar to radiolo-
could produce changes in the detection accuracy from
gists. The findings imply that in the future it should be
session to session. The results of the multiple sessions
possible to use nonradiologists in such studies if the
for the five radiologists evaluated are shown in figure 6.
task is well defined and if it involves only the detection of
Although their mean accuracies vary from 0.45 to 0.85,
a limited, specified pathologic condition.
all radiologists respond in a similar fashion to changes
in image quality. The detection accuracy of the individual Changes in Detection Accuracy
radiologist is not affected significantly by the change in
There were large differences between the detection
focal spot size for the imaging geometry used. The false
accuracies of individual radiologists (figs. 5-7). Since
positive rate tends to decrease with increasing focal spot
these variations are known to be present in all types of
size (fig. 7A).
responses from human observers, the foregoing data
Since the radiographs made with the smallest focal
can be analyzed by eliminating the individual differ-
spot tend to reproduce the fine detail in the phantom
ences. This can be done by looking at the changes in
clearly (i .e. , the amount of structured noise is increased),
detection accuracy of the radiologists as a group.
it would be expected that the radiologist might falsely
If A1 is the detection accuracy for a particular film
indicate the presence of a lesion in the noisy back-
quality and A2 is the accuracy for films of a different
ground. In addition, the radiologist would be expected
quality, the change in detection accuracy is given by the
to have a more difficult task in reading the radiographs
following formula:
made with the small focal spot because he is not accus-
tomed to viewing chest radiographs with considerable
fine detail. .A- ‘ 2 2
- (A1 + A2)/2
Since it has been shown that the root-mean-square
(RMS) noise will interfere with detection [10], we used This result is merely the change in the detection accu-
the RMS noise as a relatively simple way to characterize racy normalized to the average accuracy. The results are
fluctuations in the density of a radiograph. The RMS shown in figure 9, where the lowest noise and the
noise was determined by uniformly exposing a radio- smallest focal spot image are assigned a relative detec-
graph and measuring the density at 750 independent tion accuracy of unity. It is apparent that the variation
locations on a film (with fixed aperture of 1 mm). The due to differences between individual radiologists has
average and the square of the differences (deviations) been reduced considerably (cf. fig. 5). Increasing the
between each measurement were determined. The RMS focal spot size from 0.3 to 2.0 mm had no effect on the
noise is then the square root of the mean of the squared detection accuracy (figs. 9A and The detection
9B).

deviations or the standard deviation of the density mea- accuracy decreased for increasing RMS noise (figs. 9C
surements. The RMS noise has been discussed in detail and 9D).
in the literature [11] and recently applied in the radio- A fourth data point, X (figs. 9C and 9D), was plotted
graphic field [12]. for the Hi PIus/RP screen-film combination. Although its
It is evident (figs. 6C and 6D) that the detection value for RMS noise is not significantly different from
accuracy of the individual radiologist decreases with that of the Alpha 4/RP combination, the relative detec-
increasing RMS noise. Most radiologists tend to report tion accuracy is significantly lower, due to the lower
more false positives when the noise is increased (fig. modulation transfer function (MTF) for the Hi Plus screen
7B). Also, the radiologist tends to disagree with himself compared to the Alpha 4 screen.
 DETECTION ACCURACY IN CHEST RADIOGRAPHY 251

C A
A 0.91.
B
0.91- 3 14
34 ll
Radiologist #1 0.8 0.7L 12
0.71 1 0.6L 10
Downloaded from www.ajronline.org by 103.119.54.48 on 10/27/19 from IP address 103.119.54.48. Copyright ARRS. For personal use only; all rights reserved

IL. 8
Radiologist #3
6

3 I 4

0.61- o.4. 2

Radiol #4 J o-

O.4
#{188}%;:
1 0.41- * ‘, 0.3
Focal
1.0
Spot Size (mm)
2.0
C
I 03

0.7’- 3
0.7
1.4

1 1.2’
Radiologist #5 0.6 - I’’ fli.o.
0.51-
o.7L
1

2
0.41-
V 0.8
0.6-

Radiologist *6
0.6L 1
0.60 0.70 0.80
RMS Noise (x102)
0411 1 Fig. 7.-A, Number of false positive responses per reading session as
f &

0.3
,

1.0
I

2.0 0.55 0.60 0.66 0.70 0.75

function of nominal focal spot ,
size. B Number of false positive responses
Focal Spot Sizs 1mm) RMS Noiss )10)
per reading session as function of root-mean-square (RMS) noise. C,
Relative intraobserver disagreement as function of RMS noise.
B
0
0.9

0J 0 0.64 for all lesions over bone. (The detection accuracy

. 0.7 0.7 for those lesions on the bone edges was 0.77.) This
, 0.6 65IIultl:::::::::: 0.6 indicates that considerable improvement in detection
0.5 0.5
accuracy can be realized by making two radiographs,
both in the posteroanterior projection with the focal spot
0.4 0.4
shifted slightly vertically. About 75% of the thorax vol-
0.3 1.0 2.0 0.0 0.0 ume would then be imaged without superimposing pos-
Foal Spot 51a (mm) AMS No).. lxlO2l tenor structures, compared to about 50% of the volume
Fig. 6.-A, Detection accuracy as function of focal spot size for five
with one radiograph. However, this increase in detection
radiologists. Numbers in A and C refer to reading session . B Data from A , accuracy would require twice the x-ray dose to the
combined to show detection accuracy for individual radiologists. Num- patient and would also increase the cost of the exami-
bers in B and D refer to radiologists. C, Detection accuracy as function
of root-mean-square (RMS) noise for five radiologists. 0, Data from C nation.
combined to show detection accuracy for individual radiologists. In many respects the films used in this study were
much better than clinical radiographs. The films in this
study should be considered as optimal because (1) the
Discussion
lack of mediastinal material in the phantom reduced the
As in any study of this nature, the results must be amount of scattered radiation in the lung parenchyma;
viewed in terms of the type of radiographic study and (2) the degradation due to patient motion was absent; (3)
simulated pathology involved. In this case, the results the film densities were maintained at an ideal level,
apply only to chest radiography and to the detection of although this should be expected in the clinical environ-
relatively small lesions (6.4 mm). It is interesting that ment with adequate phototiming and quality control; and
lesions of the next smaller size (4.5 mm) were very (4) the film reading conditions were ideal. Although
difficult to detect. This tends to be supported by the using ideal films may increase the detection accuracy, it
general consensus in radiology that lesions less than 5 does not alter the conclusions of this study.
mm in diameter are virtually impossible to detect, and, The validity of such a simple detection task relative to
under normal clinical conditions, lesions larger than 5 diagnosis may be questioned. A sequential task includes
mm and closer to 1 cm are the smallest lesions normally (1) detection-something is present; (2) recognition-it
considered potentially pathologic. is definitely pathologic; (3) discrimination-it is a lesion
The overall detection accuracy for all lesions in th of a specific type; and (4) diagnosis. Thus, the detection
intercostal spaces was 0.90 compared to an accuracy of task is of prime importance, because if the object is not
 252 GRAY ET AL.

C
C
A Radiologists 1.0
A 0.8
1.0 - 0.9
0.8 0.7

, 0.7 0.6
0.9 - SEM 0.8 Radiologists M

0.8 - 0.7
0.6 . 0.5 0 T I I I
Downloaded from www.ajronline.org by 103.119.54.48 on 10/27/19 from IP address 103.119.54.48. Copyright ARRS. For personal use only; all rights reserved

D
8 0.5,: I I I 0.4
0 B All Particpants 1.0 6
All Participants

B 0 1.0 . .---.--------. 0.9 x’N.SE M

a
. 8’ 8
0.9 . ISEM 0.8

6 6 0.8 . 0.7
T I I ‘L
::4 4 0.3 1.0 2.0 0.60 0.70 0.80
0
Focal Spot Size (mm) RMS Noise )xlO”2)
12 2
E Fig. 9.-Relative detection accuracy for radiologists and nonradiolo-
z3 i I C . .
gists as function of nominal focal spot size and root-mean-square (RMS)
0.3 1.0 2.0 E noise. One standard error of mean (SEM) is indicated.
Focal Spot Size mm)
t 1.4

H 3.5

I’1 0.60
I

0.70
I

0.80
3.0

2.5

RMS Noise )10)

2.0
Fig. 8.-Detection accuracy, false positive responses, and intraob- 0
server disagreement for nonradiologists.
. 1.5
‘a
0
detected the following steps leading to diagnosis cannot -J 1.0
be carried out. Lack of detection in this study is assumed
to be attributed to two factors: (1) errors in search-the 0.5
area in which the lesion is located is not searched; and
(2) errors in detection-the area is searched but the
0.0 0.5 1.0 1.5 2.0 2.5 3.0
“signal” is below the criterion level selected by the
individual radiologist. Net Density
Several participants commented about the quality of Fig. 10.-Contrast as function of density.
the images during the study. Although they were never
aware of the screen-film combinations or focal spots personnel are trained to detect a particular pathologic
used, several radiologists commented on the excellent aspect, they will no doubt disregard other pathology that
quality of particular films which, in fact, were made with may be present in the radiograph.
the Alpha 4/RP screen-film combination (low noise, high An analysis of the location of the false positive re-
resolution [MTF]) and particularly those made with the sponses provided no additional insight into the viewing
1 .0 or 2.0 mm focal spots. One nonradiologist consist- or detection techniques of the participants. Both radiol-
ently stated that the quality of the images made with the ogists and nonradiologists recorded more false positive
Alpha 4/RP screen-film system and the 2.0 mm focal responses on or near the major vessels. Only if the
spot were superior to all others. The Alpha 4/XM screen- images produced a large number of false positive re-
film system was considered unacceptable by most partic- sponses, such as in the Alpha 4/XM screen-film combi-
ipants, regardless of the focal spot used. The Alpha 4/ nations, did a significant portion of the false positive
XM radiographs had an excessively mottled appearance responses occur in the lung parenchyma distal to the
and gave the impression of low contrast, although in the major vessels.
density range of interest the contrasts of the four screen- Significant numbers of false negative responses were
film systems were similar (fig. 10). also associated with the major vessels. In many cases
The fact that the radiologists and nonradiologists re- such lesions appeared only as a slight (photographic)
sponded similarly to changes in image quality is not density decrease in the vessel area with no lesion border
surprising, since the task involved was strictly detection or outline visible.
of a very limited “pathology.” In addition, the fact that The results of this study indicate that: (1) changes in
the nonradiologists’ detection accuracy was similar to detection accuracy should be determined for radiolo-
that of the radiologists is interesting in terms of the gists as individuals, not as groups; (2) radiologists and
training of paramedical personnel to screen radiographs. nonradiologists respond to changes in image quality in a
However, it must be remembered that if paramedical similar manner; (3) the detection accuracy is not affected
 DETECTION ACCURACY IN CHEST RADIOGRAPHY 253

by changes in the focal spot size, but the false positive sheet. And remember, the 72 radiographs must be read in less

rate decreases with increasing focal spot size; (4) the than 1 hour and 15 minutes.
detection accuracy decreases for increasing root-mean- Are there any questions?
square noise; and (5) the false positive rate and intraob-
REFERENCES
server disagreement increase with increasing root-mean-
square noise. 1 . Feddema J, Botden PJM: Adequate diagnostic information,
Downloaded from www.ajronline.org by 103.119.54.48 on 10/27/19 from IP address 103.119.54.48. Copyright ARRS. For personal use only; all rights reserved

in Television in Diagnostic Radiology, edited by Moseley

ACKNOWLEDGMENTS RD Jr, Rust JH, Birmingham, Ala., Aesculapius, 1969, pp
15-25
We thank Dr. Edward C. Shaffer, 3M Company, St. Paul,
2. Goodenough DJ, Rossmann K, Lusted LB: Radiographic
Minn., who supplied the rare earth screens and film; Stephen
Kelly, Mead Technology Laboratories, Dayton, Ohio, who applications of signal detection therapy. Radiology
105:199-200, 1972
scanned the edges to provide the microdensitometric data for
3. Goodenough DJ, Rossmann K, Lusted LB: Radiographic
determining the modulation transfer functions; and Dr. Russell
Holland, E. I. du Pont de Nemours & Co., Wilmington, Del., for applications of receiver operating characteristic (ROC)
his assistance in determining the root-mean-square granularity
curves. Radiology 1 1 0 : 89-95, 1974
4. Metz CE, Goodenough DJ, Rossmann K: Evaluation of
of the radiographic images. In addition, we thank Dr. Harold E.
receiver operating characteristic curve data in terms of
Johns for his helpful discussions, comments, and criticisms.
information theory, with applications in radiography. Ra-
diology 109 : 297-303, 1973
APPENDIX
5. Starr SJ, Metz CE, Lusted LB, Goodenough DJ: Visual
The following instructions were provided to each subject by detection and localization of radiographic images. Radiol-
means of a tape recording: ogy 116:533-538, 1975
The study in which you are about to participate has been 6. Kundel HL, Revesz G: Lesion conspicuity, structured noise,
designed to provide basic data concerning the detection accu- and film reader error. Am J Roentgenol 126:1233-1238,
racy of chest radiographs made under known, controlled con- 1976
ditions. You will be shown a set of nine radiographs for which 7. Brogdon BG, Moseley RD, Kelsey CA, Hallberg JR: Percep-
you will be asked to locate as many lesions as possible and tion of simulated lung tumors. Paper presented at the
indicate their location on the reproductions of the radiograph annual meeting of the Association of University Radiolo-
provided. After each of these nine radiographs you will be gists, Kansas City, Kan., April 1977
shown the correct lesion locations. 8. Gray JE: Edge gradient analysis in medical imaging. Sub-
After the completion of the first set of radiographs you will be mitted for publication
shown a total of 72 radiographs that must be read in less than 1 9. Gray JE, Trefler M: Phase effects in diagnostic radiological
hour and 15 minutes. You will be asked to locate as many images. Med Phys 3 : 195-203, 1976
lesions as possible, but you will not be advised if you are 10. Pollehn H, Roehrig H: Effect of noise on the modulation
locating the lesions correctly. In addition, to assure that each transfer function of the visual channel. J Opt Soc Am
answer sheet is scored properly, you will be asked to record the 60:842-848, 1970
number of the radiograph on the sheet in the upper left corner. 1 1 . Dainty JC, Shaw R: Image Science: Principles, Analysis and
In summary, there may be as many as nine lesions visible in Evaluation of Photographic-Type Imaging Processes . New
each radiograph. Accurately locate as many lesions as possible York, Academic Press, 1974
and indicate their position on the answer sheet. Be sure to 12. Barnes GT: The dependence of radiographic mottle on
record the radiograph number on the upper left corner of the beam quality. Am J Roentgenol 127:819-824, 1976