Outline

• Audiometry continued
• Review:
• Stimulus Response Matrix
• Sensitivity d’
• Bias β
• Receiver Operator Characteristic ROC
• Signal Classification Systems
• Human in the Loop
• Signal Detection in Complex displays
• When to Automate
 Hearing testing: Audiometry

©Maica, Getty Images

https://www.youtube.com/watch?v=7RGptVMv6zQ

https://www.grason-stadler.com/solutions/software/gsi-amtas?utm_campaign=GSI
%20AMTAS&utm_source=ppc&utm_medium=GSI%20Adwords

https://www.shoebox.md/the-ipad-audiometer/?
gclid=EAIaIQobChMI0Niu6pX_3wIVjh6tBh1g_wNkEAAYASAAEgJ8u_D_BwE
Audiogram showing Hearing Loss

http://entcet.com/ears_hearing/audiology/
understanding_hearing_hearing_loss_and_the_audiogr
am.aspx
 Review
Stimulus Response Matrix
Actual State of the World
Signal Noise
Yes Hit False Alarm
Interpretation
No Miss Correct
Rejection

P(H) + P(M) = 1
P(FA) + P(CR) = 1
 Exercise
Create your own Stimulus Response Matrix !

• One partner gives a series of 8 trials to the other

• Hiding the tip of the pen, either draws a happy face or pretends
• Randomly present either happy face or control trial
• Total of 4 trials of each
• Record: Trial number, Happy face Y/N, Correct Y/N
• Switch roles
 Stimulus Response Matrix
Actual State of the World
Signal Noise
Yes Hit False Alarm
Interpretation
No Miss Correct
Rejection
Probability of Detection

Wickens and Hollands, 2000

 Sensitivity d’
High
Sensitivity

Low
Sensitivity

Wickens and Hollands, 2000

Includes signal to noise ratio and operator sensitivity

 Signal Detection Bias β
P(X/S) a
β= P(X/N) = b

Conservative decision bias

Say “No” more often

b a
a
b

Say “Yes” more often

Risky decision bias

Receiver Operating
Characteristic
Signal Noise
Yes P(H) P(FA)
No P(M) P(CR)
 Optimal β
P(N)
βoptimal = P(S)

Considering the costs and values of a Decision Outcome

P(N) V ( CR ) + C ( FA )
βoptimal = P(S)
x
V(H)+C(M)
 Evaluation of Test
Accuracy = Sensitivity P(A) + Specificity P(N)
Actual Signal Noise
(Disease or (Normal)
Predicted Abnormal)
Yes Hit False Alarm
(TPF = S+) (FPF)
= Sensitivity P(A)

No Miss Correct
(FNF) Rejection
(TNF = S-)
= Specificity P(N)
 Definitions
• Accuracy: Ability to distinguish noise and
signal
• Sensitivity: Ability to distinguish signal
• Specificity: Ability to distinguish noise

https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4614595/
 Application
• Testing classification algorithms
• Comparison of different diagnostic tests
• Comparison of different test equipment or
technology

ROC shows which test is affected by operator criteria,

and which produces greater sensitivity
 Human-In-The-Loop Systems
Black Box ?

Sensory
Inputs

Decisions Motor Outputs

Feedback
Human-In-The-Loop Systems

Wickens and Hollands, 2000

 Food for thought
It is up to us to decide what human means, and exactly how it is different from machine,
and what tasks ought and ought not to be trusted to either species of symbol-processing
system. But some decisions must be made soon, while the technology is still young.
And the deciding must be shared by as many citizens as possible, not just the experts.

In that sense, the most important factor in whether we will all see the dawn of a
humane, sustainable world in the twenty-first century will be how we deal with these
machines a few of us thought up and a lot of us will be using.

- Howard Rheingold, 2000

Complex Interfaces

http://www.hcibook.com/e3/scenario/nuclear/
Manufacturing Inspection

https://www.vision-systems.com/boards-software/article/14039585/g4-
technology-ai-defect-inspection-system-analyzes-pcb-defects
Other Examples of Detection in HCI

Fault Detection Pattern Recognition

http://www.aso.com/
i.aso3/mmg.jsp?
iaso3sid=1&searchid=7845
057&typeid=1
http://rodrigo.typepad.com/
english/2005/09/
google_earth_le.html
 When to Automate ?
The Human is better for: Design
• Detection of low level signals against noisy background
Exploration • Sensing unusual and unexpected stimuli
• Adapting to the environment Uncertainty
Research • Utilizing varied experience
•
Problem solving
Making subjective evaluations

The Computer is better for:

Monitoring
• Detection of stimuli outside of human range
• Sensing specified events or targets Consistency
• Operating without fatigue or attention effects
• Storing large quantities of information
• Making objective evaluations Accuracy
Complex Calculations