Toward a Science of Robotics: Goals and Standards for Experimental Research

Leila Takayama Human-Robot Interaction Research Scientist

Scientific Principles
•Hypothesis testing •Observable, empirical, and measurable evidence •Reliable •Reproducible and falsifiable

Science & Technology

Variable-based Research
X→Y

“analog, keyboard and receiver input, high fidelity, two-way processing technologies”

Examples of Variables
•System complexity •Interactivity •Similarity to humans •Task types: collaborative, competitive •Synchronicity of interaction •Collocation

Study 1 Self Extension into Robots

Research Questions
– What aspects of human-robot interface design affect feelings of attachment, trust, control, responsibility, and agency in human-robot interaction? – Does one’s sense of self-extension increase when a robot is built by the user? – Does one’s sense of self-extension differ between more or less anthropomorphic robots?

Manipulations
Factor 1 (between): car vs. robot Factor 2 (between): Use of their assembled robot/car or a preassembled robot/car Factor 3 (between): assembly vs. no assembly

Hypotheses
Robot form Humanoid form is a strong cue of identity
– H1. People will self extend more into the car robot than the humanoid.

People extend their positive self concept into self-extended objects
– H2. People will prefer the personality of the car robot over the humanoid.
9

Hypotheses
Robot assembler
Building an object promotes self extension – H3. People will self extend more into a robot they assemble than a robot assembled by another. People extend their positive self concept into self-extended objects – H4. People will prefer the personality of a robot they assemble over a robot assembled by another.

10

Procedure
• Participants fill in pre-questionnaire • Participants given instructions and diagrams on how to assemble robot • Participants assemble robot
– (M=9 min 49 sec, SD=3 min, 34 sec)

• Participants turn on robot and test it
– Tethered control with on/off button

11

Task game
• • • • •

Procedure

Goal is to collect most points in 10 minutes Bombs sometimes explode when touched Bomb detonations deduct 30 seconds Bomb number and time controlled Questionnaire

12

Assembler Manipulation
•Manipulating assembler
– Self: Built a robot, operated same robot – Other: Built a robot, participants told they needed to operate a different, identical robot

•In fact, all participants operated the robot they built
13

Robot Form Manipulation

Measures: Self extension
Trait overlap – Personality similarity of self and other
•Galinsky and Moskowitz
– Overlap in concepts of self and human other

•Kiesler and Kiesler
– Self extension into objects

Self

Other

15

Measures: Self extension
Determining trait overlap
– Thirty item modified Wiggin’s personality test
•Completed by participants about themselves before task •Completed by participants about robot after task

– Delta of items calculated, summed to index
•(Cronbach’s α=.86)

– Smaller scores indicate greater overlap of concepts of self and robot
16

Measures: Self extension
Self reports
• 10 point scales asking about “the device you guided through the minefield” • Robot control (α=.83)
– Who was more responsible for your general performance on this task? – Who had more control over your general performance on this task?

• Sense of team
– “I felt that the robot and I were a team.”
17

Measures: Robot personality
•Robot friendliness
– Nine item index (α=.90) – cheerful, enthusiastic, extroverted

•Robot integrity
– Five item index (α=.73) – Honest, reliable, trustworthy

•Robot malice
– Five item index (α=.74) – Dishonest, unkind, harsh
18

Results: Self extension
Trait overlap

F(1, 52)=4.04, p<.05, partial η²=.13

Greater trait overlap with car robots than humanoids
19

Results: Self extension
Robot control

F(1, 52)=5.47, p<.05, partial η²=.10

Greater relative control attributed to humanoids than to cars 20

Results: Self extension
Sense of team

F(1, 52)=8.34, p<.01, partial η²=.14

Self-assembly participants felt more like a team with the robot than did other-assembly participants
21

Measures: Robot Personality
Robot friendliness

F(1, 52)=4.25, p<.05, partial η²=.08 η²

F(1, 52)=4.23, p<.05, partial η²=.08 η²

Car robots were friendlier than humanoids

Self-assembled robots were friendlier than robots assembled by others
22

Measures: Robot Personality
Robot integrity

F(1, 52)=4.20, p<.05, partial η²=.08

Car robots were rated as having more integrity than humanoids
23

Measures: Robot Personality
Robot malice

F(1, 52)=8.94, p<.01, partial η²=.15 <.01, η²

F(1, 52)=4.78, p<.05, partial η²=.08 η²

Humanoid robots were more malicious than cars

Robots assembled by others were more malicious than self-assembled robots

24

Summary of Results
H1. People will self extend more into the car robot than the humanoid. H2. People will prefer the personality of the car robot over the humanoid.

25

Summary of Results
H3. People will self extend more into a robot they assemble than a robot assembled by another. H4. People will prefer the personality of a robot they assemble over a robot assembled by another.

26

Design Implications
Goal-specific guidelines
– No form, assembly experience is uniquely optimal – Desirability of self extension informs design

27

Design Implications
•When self extension is desired
– Tele-operated robots as media, human representations
•Medical care, remote therapy

– Non-humanoid form – Promote pre-mission interaction
•Assembly, customization

28

Design Implications
•When self extension is undesirable
– Robots in hostile environments, likely failures
•Search and rescue

– Humanoid form – Minimize pre-mission interaction
•Identical but different robots
– Change robot’s name

•Altered robots
– Change voice, appearance
29

Limitations and Next Steps
• Broader population • Outside the lab • Using other robots • Long-term interactions • Long-term effects • Balancing needs of people operating and encountering robot
30

Study 2 Disagreeing Robots

Why would a robot ever disagree with a person?

Research Questions
•What influences an interface’s point of interaction? Body location? Voice location? •(How) do politeness strategies from human-human interaction inform human-computer interaction?

Design Questions
•What influences a robot’s point of interaction? •Where should speakers be placed? •(How) can computer agents influence human decisions, using effective politeness strategies?

Hypotheses
H1. People will change their decisions more often when the robot disagrees with them than when it always agrees with them, even with identical substantive content. H2. People will feel more similar to (H2a) and more positively toward (H2b) the agreeing robot than the disagreeing one. H3. A disagreeing voice coming from a separate control box will be more acceptable than a disagreeing voice that came from the robotic body.

Study Design (N=40)
Betweenparticipants Disagree 0% Disagree 60% Voice location: on robot Voice location: in box



20 men and 20 women, balanced across conditions

Procedure
• Write down decisions about desert survival items • For each item – Tell robot which item to retrieve – Robot responds with survival item information and judgment about decision – Tell robot which item to retrieve • Write down final decisions about survival items
– These final ratings will be “evaluated”

• Fill out paper questionnaire

Desert survival task
You are one of the members of a geology club that is on a field trip to study unusual formations in the New Mexico desert. It is the last week in July. You have been driving over old trails, far from any road, in order to see out-of-the-way formations. At about 10:30 A.M. the specially equipped minibus in which your club is riding overturns, rolls into a 20-foot ravine, and burns. The driver and professional advisor to the club are killed. Both of you are relatively uninjured…

Desert survival task
Rank the following items according to their importance to your survival, starting with 1 for the most important one and proceeding to 12 for the least important one. ______ ______ ______ ______ ______ ______ ______ ______ ______ ______ ______ ______ magnetic compass 20-ft by 20-ft piece of heavy-duty, light-blue canvas book, Plants in the Desert rearview mirror large knife flashlight (four-battery size) one jacket per person one transparent, plastic ground cloth (6-ft by 4-ft) per person .38-caliber loaded pistol one 2-quart plastic canteen per person, full of water accurate map of the area large box of kitchen matches

Experiment Set-up

Manipulation: Robot disagreement
Statement 1. Description of selected item 2. Judgment: disagreeing or agreeing 3. Description of alternative item 4. Request for final selection Examples The knife could be helpful in cutting down stakes to build a solar still or to build shelter. It could also assist in cutting down firewood for a fire. That is not as That is a better good as… or choice than… The pistol, which could be good for signaling for help. It could provide an alternative noise source if your voice is weak due to dehydration. Which do you choose?

Measures
Behavior • Number of decisions changed Attitudes • Perceived agreeableness of robot • Perceived similarity of robot to self • Liking of the robot
(4 items, Cronbach’s α=.94) (8 items, Cronbach’s α=.75) (2 items, Cronbach’s α=.69)

Perceived robot agreeableness

Perceived similarity to robot

People changed their minds

People like disagreement to come from elsewhere

Checking against hypotheses
H1. People will change their decisions more often when the robot disagrees with them than when it always agrees with them, even with identical substantive content. H2. People will feel more similar to (H2a) and more positively toward (H2b) the agreeing robot than the disagreeing one. H3. A disagreeing voice coming from a separate control box will be more acceptable than a disagreeing voice that came from the robotic body.

Theory-oriented Interpretations
•Politeness: distancing •Disembodiment •Perceived source
– Two separate agents: Thinker and doer – Single distributed agent

Design-oriented Implications
• • • • • Voices can be more evocative than robot bodies Agents can be sources of judgment and opinions People are sensitive to disagreements Disagreement undermines feelings of similarity When agreeing 100% of the time, put the voice on the robot body • When disagreeing (sometimes), put the robot voice elsewhere

Validity
• face validity: how reasonable a measure seems to be for its concept • content validity: how thoroughly a measure addresses the breadth of a concept • construct validity: how much a measure causally relates to other variables within one’s theory • external validity: how generalizable the results will be to other systems and contexts

Experiment Designs
•Keep it simple •Ceteris paribus •Random assignment to conditions •Balancing •Standardized tasks and measures •Behavioral and attitudinal measures •Sample representativeness

Stats
•Use with caution! •Especially with statistical modeling

Reporting Studies
• Research questions and hypotheses • Statistical significance • Reproducible methods • Discuss limitations • Thoroughly review related work • Reduce bias in language • Clear labeling • Define terms

Experiment Work Practices
•Pilot stimuli, measures, procedures with multiple types of pilot participants •Identifying the important variables and their relationships (grounded theory)

Sharing artifacts and code

www.willowgarage.com

Thanks!

Victoria Groom, Clifford Nass Claudia Jimenez, Alison King, Morgan Ames, Courtney Schultz, Paloma Ochi, Jessica Yuan Contact: Leila Takayama takayama@willowgarage.com

Data Frame Model

Sign up to vote on this title
UsefulNot useful