Signal Group 2

Aliluran
Alvarez
Aquino
Gonzales

Detection

Theory
The Origin of Signal Detection Theory

Its origins concentrate on three key developments:

First Development - Gustav Fechner (1860/1966)
Architect of the 2AFC signal detection framework.
he conceived of signal detection theory for the two-
alternative forced-choice (2AFC) task.
The Origin of Signal Detection Theory

Second Development - Louis Leon Thurstone (1927)

used the 2AFC task but in an effort to scale
subjective sensations associated with stimuli that
cannot be easily scaled in physical space
The Origin of Signal Detection Theory

Third Development - collection of researchers in the

early 1950s (perhaps most notably John Swets)
almost simultaneously, introduced a breakthrough
idea whose time had apparently come. That idea
was the existence of a noise distribution that,
contrary to what had long been assumed, was in
reach of conscious awareness.
 n of SDT
Definitio
Signal Detection Theory (SDT) is not merely a theory but
also a mathematical technique for analysing perceptual
performance. It originated as a model of how human
observers perform when they must detect ambiguous
visual stimuli of a certain type, such as targets on a radar
screen (Tanner & Swets, 1954).
-describes the task of sensory discrimination as
one of distinguishing between specific target stimuli and
other, irrelevant stimuli, referred to as signals and noise
respectively.
 n of SDT
Definitio

Two important internal factors influencing an observer’s

performance on a signal detection task:
Sensitivity
Criterion (or bias)

SDT provides a graphic model for understanding

performance on a signal detection task. The underlying
notion is that overt responses to stimuli are mediated by
internal responses within the participant.
SDT provides a graphic model for
understanding performance on a
signal detection task. The
underlying notion is that overt
responses to stimuli are mediated
by internal responses within the
participant.
Applications of SDT

1 The early application of SDT to studies of sensory performance

is considered a major advance in the understanding of human
perception.
2 SDT models the ability of an agent to compare and match some
given input with one or more known categories. There is no
particular reason why such a model should be confined to
treating only raw sensory features as possible inputs; the same
principle of comparing given-to-known and dealing with
uncertainty applies equally to higher-level cognitive judgments.
Applications of SDT

3 SDT methods and measures have been adopted for the study of
such diverse real-world tasks as follows:
Military target detection,
Motorists’ detection of hazards
Medical diagnosis
Diagnostic tasks in fields like information retrieval, weather
forecasting, survey research, aptitude testing, polygraph lie
detection, and vigilance.
 Relevance to Situation Awareness

SDT provides a practical tool for the analysis and understanding of

real-life perceptual judgements.
Question: Can it also be used in this way to assess situation
awareness?
We can consider this question by comparing the processes of
signal detection (as modelled in SDT) with the process of
situation assessment. The task of situation assessment is to
arrive at a mentally perceived situation with a full understanding
of its implications.
Relevance to Situation Awareness

This is obviously far more complex and abstract than perceptual

signal detection and involves acquiring information and
interpreting that information on the basis of prior knowledge
and expectations.
The degree of equivalence in the fact that both signal detection
and situation assessment involve discrimination.
 Relevance to Situation Awareness

‘Observer’ in situation assessment must be able to discriminate

between at least the following:
Valid versus invalid information
Valid versus invalid interpretations
Valid versus invalid inferences
A further parallel with signal detection is that the observer must
have some criterion for deciding how to respond when uncertain
about any of the above discriminations.
 Relevance to Situation Awareness

Parallels between the types of error that can be made in signal

detection and situation assessment.
Accepting invalid information
Rejecting valid interpretations of the situation
Such errors can arise because of the human vulnerability to
confirmation bias.
Confirmation bias can be regarded as a low criterion setting in the
observer, that is, an over-willingness to accept stimuli as evidence
for a particular situation.
Application to Situation Awareness Pro
bes
Application to Situation Awareness Probes
SA probes are similar to tests of students’ knowledge. There are
two basic types of probes, which we can call ‘supply’ probes and
‘selection’ probes.
With selection probes, in contrast, the correct information is
presented to the participant along with one or more incorrect
options; the participant is asked to select the correct one. An
example of a selection probe technique is the use of multiple-
choice questions. This method is embodied in what is probably
the most well-established SA probe technique, the Situation
Awareness Global Assessment Technique (SAGAT; Endsley, 1995).
Application to Situation Awareness Pro
bes
In human factors research, the cognitive content of SA has
typically been assessed using some kind of hit rate, that is, the
proportion of probes that are responded to correctly.
While this seems an obvious statistic to use in terms of face
validity, on its own it is an inadequate index of SA accuracy for two
reasons:
a participant’s hits disclose nothing about his or her false
perceptions or false beliefs about the situation; nor does it
reveal their correct awareness of what is not the case
hit rate alone fails to provide a full picture of the participant’s
awareness
Practical Example

We have so far applied this technique (dubbed QUASA, for

Quantitative Analysis of Situation Awareness) to SA probe
data obtained from a small number of military trials of
varying size. The most significant of these has been a multi-
national experiment held in February 2003. Led by the U.S.
Joint Forces Command, LOE2 (the second in a series of
Limited Objective Experiments) involved five nations plus
NATO collaborating via the Collaborative Federated
BattleLab Network, a secure online environment designed
to facilitate allied experimentation.
SA Probes in LOE2

One aim of the LOE2 analysis activity was to focus on the

human issues of situation awareness and shared
awareness. To this end, 58 players (located in five
countries) were asked to answer SA probe statements at
2-hourly intervals in order to provide a measure of their
awareness of the current situation.
SA Probes in LOE2

As an example of a probe statement:

The Commander of the […] Air Force has
recently resigned over corruption charges.
Equal numbers of true and false probe
statements were carefully formulated, with
the probe construction process going
through several iterations. During the
experiment, five probes were presented to
all participants every two hours.
SDT in Medical

For example, when doctors check a

patient’s hearing capabilities, they emit
different pitches of sound (the signal)
and expect the patient to identify its
presence. The patient must raise a
hand whenever they hear the noise
(the response).
SDT in Medical
Patient-Device Interaction
The signal is a bodily output from the
patient and the response is the alarm
that sounds on the device. In between
the bodily output and the alarm, the
device must process the information
and decide if it is in the designated
parameters.
SDT in Medical
Device-Clinician Interaction
Patients in a hospital may be
connected to a myriad of devices in
addition to cardiac monitors, such as
monitors for vitals, ventilators, infusion
pumps, blood pressure monitors, etc.
These devices have auditory outputs
which are the signals in this signal
detection interaction.
SDT in Law Enforcement
In law enforcement, Signal Detection
Theory can be applied in police
lineups. The crime victim or witness's
putative identification of a suspect is
confirmed to a level that can count as
evidence at trial.
Example:
Question: Can you confirm the
person to be the perpetrator?
SDT in Law Enforcement

Another example is the use of SDT in

behavioral analysis. The FBI uses indicators
and parameters to determine whether or
not an individual is predisposed to criminal
behavior. Profilers decide the likelihood of
an individual to become a suspect through
extrapolation of evidence. Thus, they are
the observers in this situation.
Example:
Question: Does the individual fit the
profile
1 = Correct Rejection
2 = Hit
3 = Miss
4 = False Alarm
Sensitivity and Response Bias

1. Sensitivity (d')
Sensitivity is an index of the human's ability to
distinguish the signal (e.g., radio communication, radar
signal, other aircraft) from noise (e.g., static, engine noise).
It can be determined by properties of the receptor
system, by the skill of the observer, and by properties of
the environment or communication channel (e.g., amount of
background noise).
Formula for d':
d' = z (FA) – z (H)
Where: FA = False Alarm
H = Hits

d' is the distance from the noise curve to the signal

curve as indicated in the picture.
Ex. 1 Find d’ for Liz who had a hit rate of 0.90 and a false
alarm rate of 0.30.

Ex. 2 Calculate d’ for Bob who had a hit rate of 0.68 and
false alarm rate of 0.09.
1 Liz and Bob are virtually identical in their sensitivity. Liz was
more willing than Bob to say ‘Old,’ which resulted in her higher
hit rate and higher false alarm rate.
The higher the sensitivity, the more likely the observer can
detect the stimulus; that is, the probability of HITS and
CORRECT REJECTIONS will be high, and the probability of
MISSES and FALSE ALARMS will be low.
Sensitivity and Response Bias

1. Response Bias (β)

Response bias is an index of the human's decision-
making criterion; that is, the human's tendency to respond
“yes” or to respond “no.”
Response bias will be a function of the value of each
of the four alternative outcomes and the expectancy or
likelihood of a signal being present.
Classification of Response Criterion

Liberal - minimizes missed detections but increases

exposure to false alarms.
Conservative - minimizes false alarms but increases
exposure to missed detections.
Neutral - do not favour certain regions beyond what is
necessary to eliminate or reduce regional disparities.
 When there’s uncertainty, the observer leans more
towards saying yes to a signal. As a result, there will be more
false alarms than misses.
 When there’s uncertainty, the observer leans more
towards saying no to a signal. As a result, there will be more
misses than false alarms.
 This type of response does not necessarily lean towards
a certain side, thus, the rate of which we can have false alarms
is equal to the rate of which we can have misses.
 The computed Criterion is the z-score of this stimulation
value on the Signal Absent distribution.
Thus, Criterion = z(FA)
Ex. Bob had a hit and false alarm rates of 0.68 and 0.09,
respectively. What is his d’ and his Criterion?
Per Z score table:
0.68 = -0.47 while 0.09 = 1.34
d' = 1.34 – (-0.47)
d' = 1.81
Answer: Bob has d’ = 1.81 and Criterion = 1.34.
Adjusting the Response Criterion

Trade-off - balancing of factors all of which are not

attainable at the same time
Ex. In high-traffic areas the presence of other aircraft is
more likely and detecting them is very important. Thus, in
this phase of a mission the probability of detecting other
aircraft when they are present (HITS) will be high. However,
there will also be an increased probability that a speck on
the windscreen will be mistaken as a signal (FALSE ALARM).
The criterion can also be manipulated through the use of
payoffs (e.g., involving incentives for any of the specified
outcome).
Adjusting the Response Criterion

When a signal is important and likely, then humans should

adjust the response bias so that the probability of HITS and
FALSE ALARMS will be relatively high and the probability of
MISSES and CORRECT REJECTIONS will be relatively low.
When a signal is not important and when it is unlikely, the
response bias should be adjusted so that the probability of
HITS and FALSE ALARMS will be relatively low and the
probability of MISSES and CORRECT REJECTIONS will be
relatively high.
Receiver Operator Characteristic (ROC)

A Receiver Operator Characteristic

(ROC) curve is a graphical plot used
to show the diagnostic ability of
binary classifiers. It was first used in
signal detection theory but is now
used in many other areas such as
medicine, radiology, natural hazards
and machine learning.
Creating a ROC Curve

A ROC curve is constructed by plotting the true positive

rate (TPR) against the false positive rate (FPR). The true
positive rate is the proportion of observations that were
correctly predicted to be positive out of all positive
observations (TP/(TP + FN)). Similarly, the false positive rate is
the proportion of observations that are incorrectly predicted
to be positive out of all negative observations (FP/(TN + FP)).
Interpreting the ROC Curve

The ROC curve shows the trade-off

between sensitivity (or TPR) and
specificity (1 – FPR). Classifiers that give
curves closer to the top-left corner
indicate a better performance. As a
baseline, a random classifier is expected
to give points lying along the diagonal
(FPR = TPR). The closer the curve comes
to the 45-degree diagonal of the ROC
space, the less accurate the test.
Area Under Curve (AUC)

To compare different classifiers, it can be useful to

summarize the performance of each classifier into a single
measure. One common approach is to calculate the area
under the ROC curve, which is abbreviated to AUC. It is
equivalent to the probability that a randomly chosen positive
instance is ranked higher than a randomly chosen negative
instance. It is equivalent to the two sample Wilcoxon rank-
sum statistic.
ABSOLUTE JUDGEMENT

Absolute Judgement occurs when an observer assigns a stimulus

into one of multiple categories along a sensory dimension.

Examples:
An inspector of wool quality who must categorize a given
specimen into one of several quality levels.
A van driver who must interpret and recognize the color of a
display symbol appearing on his map display.
QUANTIFYING INFORMATION

Bit – unit of information, stands for binary digit, coinage by John

Tukey at IBM, which was taken up by Claude Shannon who was the
founder of modern information theory.

A bit measures the distinction between two possibilities, which are

famously called 0 and 1. Also, it can be any two possibilities like
true and false, yes and no, hot and cold, etc.
SINGLE DIMENSION

In a one dimensional absolute judgement task, a

person is presented with a number of stimuli that
vary on one dimension (e.g., 10 different tones
varying only in pitch) and responds to each stimulus
with a corresponding response.
Performance is nearly perfect up to 5 or 6 different
stimuli but declines as the number of different
stimuli is increased.
Limit Stimuli for the Worker
Channel Capacity ( Experimental Results)

George Miller discussed between the limits of one

dimensional absolute judgement and the limits of short
term-memory.
The information contained in the input can be
determined by the number of binary decisions that
need to be made to arrive at the selected stimulus and
the same holds for the response.
Channel Capacity ( Experimental Results)

Therefore, people’s maximum

performance on one-dimensional
absolute judgement can be
characterized as an information channel
capacity with approximately 2 to 3 bits
of information, which corresponds to the
ability to distinguish between four and
eight alternatives.
-Errors began to occur (HT < HS)
George Miller
(1920-2012)
Cognitive Psychologist
Organize Information Channel
Edge Effect (Experimental Results)

Stimuli located in the middle of the range of presented

stimuli are generally identified with poorer accuracy
than those at extremes (Shiffrin & Nosofsky, 1994)
Example: Mild cases of defects are less recognizable
than severe ones.
MULTI-DIMENSIONAL JUDGMENT

Most of our recognition is based on the identification

of some combination of two or more stimulus dimensions
rather than levels along a single dimension.
MULTI-DIMENSIONAL JUDGMENT

Orthogonal Dimensions
The level of the stimulus on one dimension
can take on any value, independent of the
other.
Example: Weight and hair color
As more dimensions are added, more total
information is transmitted, but less
information is transmitted per dimension.
Orthogonal dimensions maximize Ht, the
efficiency of the channel.
MULTI-DIMENSIONAL JUDGMENT

Correlated Dimensions
The level on one constrains is the level of one
another.
Example: Height and Weight
As more dimensions are added, the security
of the channel improves, but Hs limits the
amount of information that can be
transmitted.
Correlated dimensions minimize Hloss; that
is, they maximize the security of the channel.
Thank
you!
Have a
great day
ahead.