INFRAREDPHYSICS

&TECHNOLOGY
ELSEVIER Infrared Physics & Technology38 (1997) 373-381

Influence of sensor motion on infrared target acquisition

O. Hadar, A. Mandelblat, R. Sabath, N.S. Kopeika, S.R. Rotman *
Department of Electrical and ComputerEngineering, Ben-Gurion Universityof the Negev, 84105 Beer-Sheva, Israel
Received 16 January 1997

Abstract

The effect of linear image motion and high frequency vibrations on human target acquisition performance is considered.
Two clutter metrics, one local and the other global, are combined to form a single signal-to-clutter ratio (SCR) metric. The
SCR is used as a parameter in the model for actual target acquisition results. Two experiments involving human observers
are considered: (1) A static experiment is performed with spatial filters whose frequency characteristics are those appropriate
for the image degradation obtained from a moving sensor, and (2) a dynamic experiment is executed which imitates the
operation of a scanning camera with a constant velocity as in airborne imagers. It was found in both experiments that image
motion increases the detection time of a target by the observer. As the complexity of the original images increases, detection
time was found to increase more rapidly as a function of blur radius in the first experiment and velocity in the second
experiment. However, our blurred images in the first experiment, representative of the degradation expected from motion,
actually enables more observers to eventually acquire the target despite the longer acquisition time. 1997 Elsevier Science
B.V.

1. Introduction gets. Two experiments that imitate the phenomena of

image motion degradation were performed to obtain
The human visual system and brain amaze us with
the new data. The first experiment (the static experi-
their ability to compensate for motion and thus per-
ment) uses still images that are spatially degraded to
mit the effective detection of targets from various
simulate image motion. The degradation is obtained
backgrounds. This ability is influenced by several
by using spatial frequency filters that represent the
external factors, such as: image resolution, varying
reduction of resolution. Two kinds of image motion
backgrounds, distance of the observer from the dis-
are represented here, linear image motion and high
play screen and duration of target exposure. One of
frequency vibrations. The second experiment is a
the important factors that influences the ability to
dynamic one, where the images move on the screen
detect a target is the relative velocity between the
to imitate a scanning camera with a constant veloc-
scene and the sensor. The motivation of the research
ity. The measured parameters common to these ex-
presented here is to develop new data on the re-
periments are (1) the detection time (t a) of a target
sponse of the human visual system to relative motion
by the observer, and (2) the probability of detection
between itself and images that include military tar-
in 'infinite time' (P=). These two parameters are
measured from the observer's reaction, and they are
* Corresponding author. E-mail: srotman@bguee.bgu.ac.il. compared to metrics related to the signal-to-clutter

1350-4495/97/$17.00 1997 Elsevier Science B.V. All rights reserved.

PII S 13 50-4495(97)00023-6
374 O. Hadar et al. / Infrared Physics & Technology 38 (1997) 373-381
ratio (SCR) in the image, a metric specifically devel-
oped for this paper. The importance of this study lies
in the possibility of using its results to predict the
performance of the human visual system regarding
actual military targets in real situations.
The purpose of the analysis here is to investigate
how t d and P~ change for different kinds of image
motion and different blur radii. This kind of analysis
can be very helpful for estimating the ability to
automatically extract a target from its background
under image motion blur conditions. It can also be
useful for estimating the probability of a human's
detection of a target in scenarios that involve image
motion.
2. Thermal imaging model for target acquisition
In this section, only the final result for the proba-
bility of acquiring a target in thermal images is
considered; the dynamic search model appears in
Ref. [1]. The probability of detection P(t) as a
function of time t is given by the following equation
for not overly bright targets,
P(t)=P~[1- [ P~t]] (1)
Eq. (1) is part of the standard military search model
for infrared target detection developed in the Night
Vision and Electronic Sensors Directorate (NVESD).
The term P~ in Eq. (1) represents the fraction of the
normal observer ensemble that can successfully de-
tect the target in 'infinite time'. P~ is also the static
probability of detection when the position of the
target is known.
3. Metrics of clutter
In this section, several metrics that distinguish
between target and background are considered. These
metrics are related directly to the human vision
process of detecting targets.
We have been developing metrics to evaluate
clutter that are correlated to the human visual sys-
tem. One of these methods is based on the relation-
ship between the human detection system and the
statistics of the pictures. We start by dividing the
picture into blocks roughly twice the size of the
expected target. A Difference of Offset Gaussian
(DOOG) filter is applied to the picture and the
resulting picture is thresholded; this imitates the
human preattentive visual system. Then the number
of edge points are counted in each block. A function
of this is used as our metric; the technical details of
this metric are considered in Ref. [1]. This metric is
considered as a global parameter because the whole
image, not just the target area, is processed. The
result of this metric is given by
POE = M E POE/2.r, (2)
i=l
where POEi, r is the fraction of the edge points that
pass the threshold in each separate block. Other
possible clutter metrics can be found in Refs. [2,3].
We also have developed a local parameter for
measuring the degree of target clarity. This metric is
called target complexity (TC), and it is proportional
to the difference between the target and background
histograms. The details of the TC metric can be
found in Ref. [ 1].
From the metrics described above it is possible
now to define a signal to clutter ratio (SCR). It is
given by
SCR=TC/POE. (3)
The logic behind this choice is that the TC indi-
cates how prominent the target is on its background,
while the POE represents the complexity of the
background (clutter) which makes the target more
difficult to detect.
All the metrics described above were calculated
for the 60 images used in the experiment, of which
36 are blurred images. The purpose of using these
metrics is to enable us to compare the results from
all the images as one group. These metrics indicate
the complexity of each image, where this complexity
is influenced not only by the scene itself but also by
the blur induced from image motion. Based on checks
during our experiment, we believe that the conclu-
sions of our study are independent of the choice of
clutter and target metrics; any from the Refs. [1-3]
would give the same results.
 O. Hadar et al./ Infrared Physics & Technology38 (1997) 373-381
4. Image motion modulation transfer function
(MTF)
Two kinds of image motion are considered in the
static experiment: linear motion and high frequency
vibration (see Fig. 1). To simulate image motion in
the analysis, the source image is first passed through
a synthetic filter that represents each of these mo-
tions. The method of deriving the image motion
optical transfer function (OTF) is described else-
where (Ref. [4]). Conceptually, in this technique the
image motion line spread function (LSF) is equal to
the image motion histogram. The OTF is obtained by
the Fourier transform of the LSF. The MTF result for
linear image motion is given by
M1 = sinc(rrfd),
where f is the spatial frequency and d is the blur
radius which is equal to Vte, V being constant veloc-
ity and te camera exposure time.
The MTF for high temporal frequency vibrations
is given by
Mhf = Jo(27rfd),
where D is the amplitude of the sinusoidal vibration.
We note that for the case of low frequency vibra-
tion there is no unique OTF because the degradation
process is random and depends on the exposure time
duration as well as the sine phase at which the
exposure begins (Ref. [5]). The influence of this
d i , ~ u
LSF
1 ~ rligh
Fig. 1. Experimentalsetup for the static experiment.
375
motion is not considered here, but is to be examined
in future experiments.
5. Experiment 1: Degradation by image motion
spatial frequency f'dters
5. I. Experimental description
In the static experiment, observers were asked to
view still images degraded by image motion spatial
filters, and detect targets in the images. Observer
detection time ta was calculated as a function of the
blur radius. Three levels of blur radius are chosen:
low ( d = 2.63 pixels), moderate ( d = 5.26 pixels),
(4) and high level degradation ( d = 7.89 pixels). Our
analysis of the static experiment determines the ob-
server's ability to detect the target. In a given data
set, all the different image motions were considered
to have the same blur radius extent. Observer detec-
tion time t o was calculated as a function of the blur
radius. The images were blurred according to the
(5) method presented in Section 4 (Fig. 1). The experi-
ment was conducted in a room with natural lighting
(0.5 Ix) such that vision was photopic. The experi-
ment consisted of 60 images, 36 of which contained
targets and 24 of which did not. There were 12 types
of targets in this experiment which were shown to
the observers before the experiment. Each subject
Display
l Window
selection [
128x 128[pixels]
376 O. Hadar et al. / Infrared Physics & Technology 38 (1997) 373-381

7 , , , , J
saw each of the 60 images presented in a random
order. The distance of the subject from the monitor
screen was 50 cm. Each image was displayed for a
maximum of 30 s. The observer was asked to press 5

one of the keys on the keyboard when h e / s h e

detected a target. The detection time (t 0) was mea-
sured by the internal clock of the computer and was
saved in a file. Thereafter the subject had to specify Image # 7

the location of the target by pointing to the location o o

with the mouse. In the case of a correct identifica- 2

tion, the time of identification was recorded in a file.

1
A misidentification was rejected automatically. The
subject never received feedback as to the correctness i L I I I
of his/her response. O0 2 4 6 8 16 12
Blur radius [pixels]

Fig. 2. SCR as a function of blur radius for linear image motion.

5.2. Image data

The images that were used in our experiment are
based on 12 military images (512 512 pixels) dis-
played in Ref. [6]. The images used were low-al-
titude photographs of rural New York State with
target vehicles put in by standard digital signal pro-
cessing techniques. The size of the target is about
12 12 pixels. From these basic images, 60 subim-
ages (128 128 pixels) were produced, where 36 of
them included a target and 24 did not. The 60 images
differed from each other by using a different portion
from the original 12 images. Each image was pre-
sented only once so that the observer always saw a
different image.
function of blur radius. The blur radius was varied
from 1 pixel to 12 pixels and the SCR was derived
for each blurred image. In Figs. 2 and 3, SCR is
plotted as a function of blur radius for linear image
motion and high frequency vibrations.
It appears from these figures that the SCR in-
creases with blur radius. On first glance, since the
SCR metric functions in some way as a signal-to-
noise ratio, we may have expected SCR to decrease
with blurring. However, when we look at POE and
TC as functions of the blur radius, we see that while
TC remains fairly constant with blur, it is the clutter
content which is noticeably decreasing. Small natural

5.3. Observers

The observers that participated in the experiment

all had 2 0 / 2 0 or corrected vision. In the first experi-
ment (the static case), there were 10 participants.
The subjects were undergraduate students from the J
J
behavioral sciences who have no experience with tu J

military target acquisition. Image

5.4. SCR as a result of image motion 4 ~

The metric for SCR was measured as a function

Image # 1 o
of the blur radius for linear image motion and high o

frequency vibrations. Three representative images 2 i i i i r

2 4 6 8 10 12
(see Ref. [6] for the actual images) are evaluated Blur radius [pixels]

under image motion conditions. To illustrate effects Pig. 3, $CR as a function o f blur radius for high frequency
of image motion, we consider the values of SCR as a vibrations.
 O. Hadar et al./ Infrared Physics & Technology 38 (1997) 373-381 377

300
variations in the image background are 'washed out'
by the motion. The same result was obtained in our
0
250
previous work for ATR analysis that appears in Ref. 0
0
0
[7]. This result is not a function of the particular 0
0
o
target or clutter metric, but is rather an effect related 2OO o
o Image # 1
o
to the natural backgrounds in our scene. The reduc- ..&
o
o
o
tion of 'clutter', which is proportional to POE and is =~ 1so o
o
o
the denominator of Eq. (3), causes the SCR to o
o
o
increase for both linear image motion and high fre- 100 o
o

quency vibration [8].

5O
#7
+ + + + + + + + + + + + + + + + +
Image o
o
o
o o
o
i
5.5. Static experiment results + + + + + + + + + + 2

i i i i i r i

0.2 0.4 06 0.8 1 1.2 1.4 1.6

1~ [l/s]
Two parameters are considered: the inverse of the
average detection time ( 1 / t d) and P~. In addition to Fig. 5. Blur radius as function of 1 / t d for high frequency
Figs. 2 and 3, we show Figs. 4 and 5 where the blur vibration.
radius is plotted as a function of 1/td for linear
image motion and high frequency vibration, respec-
tively. blurred image and, therefore, it takes more time to
The conclusions for the static experiment are: find the target than in the original image.
(1) In blurred images, time for detection increased (3) Although we cannot estimate a specific func-
despite the increase in SCR. These results are con- tion relating P~ to SCR, we can still conclude that in
trary to the results for static unblurred images in Ref. general P~ also increases with blur radius. This
[1], in which td decreased with increasing SCR. The result would initially appear to be suspicious; can the
results in Ref. [1] were for unblurred images with percentage of observers detecting the target go up as
different degrees of target contrast and clutter con- the image is blurred? We refer the reader to the
tent. results reported in Ref. [9] on the same images used
(2) SCR decreases with increasing td. The task of here; the results are given in Table 1. Of the 12
acquiring the target appears to be more difficult in a images given, the higher resolution led to an increase
in P~ in 4 of the images, a decrease in 6 of the
images, and no change in 2 of the images. Evidently,
80

7(3
o Table 1
o Results from visual experiments reported in Ref. [9]
60 o Image # 1
o Image P~ low resolution P~ high resolution
o
~5C
o
1 0.33 0.5
o 2 0.75 0.58
+
4-
o 3 0.91 0.91
+ Image # 7 o
+ 4 1.00 0.9l
+ o
~30
+
+ o 5 0.91 0.91
+ o 6 0.83 0.5
+
20 + o 7 0.33 0.59
+
+ o
+ 8 0.33 0.75
10 o
+
+ o 9 0.16 0.41
+
i , i i i i i i i
10 1.00 0.91
0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1,6 1.8 11 0.33 0.09
lft [l/s]
12 0.59 0.33
Fig. 4. Blur radius as function of 1 / t d for linear image motion.
378 O. Hadar et a l . / Infrared Physics & Technology 38 (1997) 373-381
the decrease in resolution increased in many cases
the detectability of the target, much as we report
here.
(4) Conclusions (2) and (3) appear to be contra-
dictory. Detection time td increases with blur radius.
Therefore, it is expected that fewer observers will
find the target so that P~ might be expected to
decrease. However, it turns out that more observers
find the target eventually, so that P~ actually in-
creases with blur radius. A similar result was ob-
tained in Ref. [10], where it was found that the mean
acquisition score (ACQ) was uncorrelated to the
mean search time. This contradiction can be ex-
plained by separating the observer reaction time (t d)
and the number of observers that react correctly
(P~). It is possible that increasing the blur in the
image increases the time for acquiring the target. As
the blur radius increases, the averaging process over
the image increases so that it is difficult to recognize
small details. This unclear image causes the observer
to be more cautious in his/her decision of the target
location. On the other hand, the blurring effect causes
irrelevant details in the image to disappear. For a
target that is large enough it will be easier for the
observer to focus on it.
(5) Increasing the blur radius caused the average
detection time to increase~ For images with low
complexity or high SCR (before blurring), the in-
crease with t a is less noticeable than in high com-
plexity images. The explanation for this phenomenon
Zoom Image (128 X 128, pixels)
Fig. 6. Experimental setup of the dynamic experiment.
is that for 'easy images' (high SCR) the target is
more prominent against its local background so the
blurting effect is not critical for the detection pro-
cess. However, for 'difficult images' (low SCR),
even a small blur radius causes significant reduction
in the image quality so td increases significantly.
6. Experiment 2: Linear motion simulation
6.1. Experiment detection
In the dynamic experiment, twelve images were
scanned from top left to bottom right, as shown in
Fig. 6. To the subject it appeared as if the image was
moving from right to left at a constant speed. The
image seen by the subject was 128 128 pixels
where the entire image was 512 X 512 pixels. Six
discrete velocities were used, 490, 395, 240, 140, 95,
and 48 pixels/s. They were presented to the ob-
server in the order of highest velocity to lowest.
Each image was displayed 10 times before decreas-
ing the scanning velocity. This process was stopped
as soon as the observer indicated the detection of a
target. Only correct identification qualified for con-
sideration and were stored in a file similar to the first
experiment. The time of detection (the same as t a in
the first experiment) was calculated to be the cumu-
lative time that the target actually was displayed on
the screen.
Display
"x
,~Eye
 O. Hadar et al. / Infrared Physics & Technology 38 (1997) 373-381 379

6.2. Experimental results Table 2

Results for td and P~ of image #7 in experiment 2

The two measured parameters, td and P~, are Average detection time (s) P~ Velocity (pixels/s)
expressed as functions of the image scanning veloc- -- 0 490
ity for each image in the dynamic experiment. Image -- 0 395
1.9 0.23 240
#7, which is considered to be the 'difficult image'
1.04 0.846 140
(low SCR), is presented in Fig. 7. The dynamic 0.8 0.923 95
experiment measurement results td and P~ are sum- 0.6 0.923 48
marized in Table 2. In Fig. 8, td is plotted as a
function of velocity (pixels/s) for three representa-
tive images (#5, #8, and #10).
The conclusions from this experiment are:
(1) Image motion decreases the ability of the SCR) and consequently the detection time increased
observer to find the target. The detection time in- more rapidly than for the other two images. The
creases with the velocity of the camera scanning average detection time appears to change more grad-
simulation; this is logical when one considers the ually with the scanning velocity for low complexity
increased blur in the image (Ref. [11]). As the com- (high SCR) images.
plexity of the original image (before image motion) Overall, to emphasize this point, we show in
increases, the detection time becomes more influ- Table 3 the values of POE for each image, the values
enced by image motion. In Fig. 8, image # 8 is of the detection time, and the velocity for which the
considered to have the larger complexity (or lower detection time doubled. In the five images with POE

Fig. 7. Typical image with a target (image #7).

380 O. Hadar et al. / Infrared Physics & Technology 38 (1997) 373-381

Table 3
Velocity at which detection time doubled vs. POE
Image # Clutter (POE) Detection time to (s) Velocity to = 2 td (s)
at low velocity (pixels/s)
1 9.4 0.4 455
2 3.2 0.45 293
3 14.7 0.4 255
4 4.5 0.29 485
5 2.7 0.38 603
6 5.9 0.35 412
7 29.8 0.6 158
8 14.9 0.5 135
10 26 0.35 800
11 48 1.3 113
12 10.6 0.5 125

values less than 10, the velocity for doubling the 7. Conclusions
detection time was greater than 255 pixels/s (mean
of 449 pixels/s). For POE values greater than 10, The first experiment includes degradations of ru-
the velocity for doubling the detection time was less ral images by using a spatial frequency filter appro-
than 255 pixels/s with one exception (image #10) priate to two kinds of image motion, linear motion
(mean of 264 pixels/s with image #10; mean of 177 and high frequency vibrations. The ability of the
pixels/s without image #10). (We are still investi- observer was examined under different blur radius
gating why image #10 is an exception.) conditions. Image motion degradation was found to
(2) A similar conclusion can be derived for P~. increase the SCR, and more observers were able to
As the complexity of the original image increases eventually acquire the targets. However, ta increases
(SCR increases), the reduction of P~ is more promi- with the blur radius. These results are contrary to the
nent with increasing velocity. For image #8, P~ results for static images appearing in Ref. [1].
reached zero for a velocity of 400 (pixels/s). In the second experiment, image motion increased
the detection time and decreased the probability of
3
detection. High clutter content notably increased the
negative effect of image motion.
2.5

References
2 o - image # 10

g x - Image # 5
[1] G. Tidhar, G. Reiter, Z. Avital, Y. Hadar, S.R. Rotman, V.
~ 1.5
+ - image # 8
George, M.L. Kowalczyk, Modeling human search and target
acquisition performance: IV. Detection probability in clutter
E
environment, Opt. Eng. 33 (3) (1994) 801-808.
[2] D.E. Schmieder, W.R. Weathersby, Detection performance in
clutter with variable resolution, IEEE Trans. Aerospace Elec-
+,.t
0,5 tron. Syst. 19 (4) (1983) 622-630.
[3] J.M. Cathcart, T.J. Doll, D.E. Schmieder, Target detection in
i i i t i i i | i
urban clutter, IEEE Trans. Syst. Man Cybernet. SMC-19 (5)
50 1O0 150 200 250 300 350 4go 450 500 (1989) 1242-1250.
Velocity [pixel/s]
[4] O. Hadar, I. Dror, N.S. Kopeika, Image resolution limits
Fig. 8. Average detection time to is plotted as a function of resulting from mechanical vibrations: Part IV. Real-time
velocity (pixels/s) for three representative images (#5, #8, and numerical calculation of optical transfer functions and experi-
#10). mental verification, Opt. Eng. 33 (1994) 566-578.
 O. Hadar et aL /Infrared Physics & Technology 38 (1997) 373-381
[5] D. Wulich, N.S. Kopeika, Image resolution limits resulting
from mechanical vibrations, Opt. Eng. 26 (6) (1987) 529-533.
[6] S.R. Rotman, G. Tidhar, M.L. Kowalczyk, IEEE Trans.
Aerospace Electron. Syst. 30 (1) (1994) 87-90.
[7] O. Hadar, A. Kuntsevitsky, M. Wasserblatt, N.S. Kopeika,
S.R. Rotman, Automatic Target Recognition During Sensor
Motion and Vibrations, Opt. Eng. 34 (8) (1995) 3062-3069.
[8] S.R. Rotman, G. Tidhar, M.L. Kowalczyk, Clutter metrics
for target detection systems, IEEE Trans. Aerospace Elec-
tron. Syst. 30 (1) (1994) 81-91.
[9] L.A. Scanlan, Target Acquisition Model Development: Effect
381
of Realistic Terrain, Hughes Technical Report P76-484, pp.
1-78, Man-Machine Systems Sec., Display Systems Lab.,
Radar Avionics Div. and Advanced Programs Lab., Tactical
Systems Div., Hughes Aircraft Co., Culver City, California,
1976.
[10] C.J. Woodruff, G.N. Newsam, Displaying undersampled im-
agery, Opt. Eng. 33 (2) (1994) 579-585.
[ll] S.R. Rotman, E.S. Gordon, O. Hadar, N.S. Kopeika, V.
George, M.L. Kowalczyk, Search strategy for optimal target
acquisition performance, Inf. Phys. Tech. 36 (6) (1995)
1025-1034.