High Quality
Open the downloaded document, and select print from the file menu (PDF reader required).
CIMMI - A Contextually Informed MultimodalIntegration Method for a Humanoid Robot in anAssistive Technology Context
Elizabeth Harte
Bachelor of Science (Hons.), Computer Science (2005)University of AucklandSubmitted for the Degree of Master’s of Engineering Science (Research)Intelligent Robotics Research CentreDepartment of Electrical & Computer Systems EngineeringMonash University, Clayton, VIC 3800, AustraliaApril 2008
Declaration
I declare that, to the best of my knowledge, the research described herein is originalexcept where the work of others is indicated and acknowledged, and that the thesishas not, in whole or in part, been submitted for any other degree at this or any otheruniversity.
Elizabeth HarteMelbourne, Australia April 2008
i
Acknowledgments
My thanks go to Prof. Ray Jarvis for his enthusiasm and support over the past 2years. He is an inspiring roboticist and researcher. I will be lucky to be even half ascreative as him in my life.I would also like to acknowledge Mitsubishi Heavy Industries Ltd for the uniqueopportunity to use the Wakamaru robot for this project.I would like to thank all the members of the Intelligent Robotics Research Center.I have met some great people and made some great friends. Thank you for welcomingthis Kiwi into the circle. In particular, thanks go to Alan Zhang and Jay Chakravartyfor their valuable discussions and friendship during the development of this project.Finally, I thank Rob Davidson for his moral support during this project. His en-couragement, support and editorial skills were greatly appreciated during the writingof this thesis. Kia ora.
ii
CIMMI - A Contextually Informed Multimodal Integration Method for a Humanoid Robot in an Assistive Technology Context
Info and Rating
Sections
« prev | next »- Introduction
- 1.1 Motivation
- 1.2. MULTIMODAL HUMAN-ROBOT INTERACTION
- 1.2 Multimodal Human-Robot Interaction
- 1.2.1 Modes of Input
- 1.2.2 Contextual Knowledge
- 1.2.3 Multimodal Integration
- 1.3 Contributions
- 1.4 Thesis Overview
- Literary Review
- 2.1 Service and Field Robotics
- 2.2. HEALTHCARE ROBOTICS
- 2.2 Healthcare Robotics
- 2.3. SOCIAL ROBOTICS
- 2.3 Social Robotics
- 2.4 Multimodal Integration
- 2.4.1 Modes of Input
- 2.4.2 Contextual Knowledge
- 2.4.3 Methods of Integration
- 2.5 Multimodal Human-Robot Interaction
- 3.1 Functional Scenarios
- 3.2 Software Overview
- 3.3 Speech Recognition
- 3.4 Symbolic Gesture Recognition
- 3.4.1 Segmentation
- 3.4.2 Region Tracking
- tracking and the three features specified
- 3.4.3 Static Gesture Recognition
- 3.5 Contextual Knowledge
- 3.5.1 Conversational Context
- 3.5.2 Situational Context
- 3.6 Integrating Speech and Symbolic Gestures in an In-
- 3.6.1 CIMMI Overview
- 3.6.2 The Iterative Approach
- 3.6.3 The Sequential Approach
- 3.7 Hardware Overview
- 3.8 Wakamaru’s Responses
- Experiments
- 4.1 Speech Recognition
- 4.1.1 Experiment 1: Selecting the Number of Recognition Hy-
- Results
- 4.1.3 Experiment 3: Word Recognition Rate of Parsed Speech Re-
- 4.1.4 Discussion
- 4.2 Gesture Recognition
- Segmentation Only
- and Motion Segmentation
- 4.2.4 Discussion
- 4.3 Multimodal Interaction Accuracy
- 4.3.1 Experiment 1: Accuracy of the Individual Modes
- 4.3.2 Experiment 2: Using the Iterative Approach
- 4.3.3 Experiment 3: Using the Sequential Approach
- 4.4 Object Recognition
- tion and Shapematcher in the Old Laboratory Environment
- tion and Shapematcher in the New Laboratory Environment
- of the ShapeMatcher approach
- 4.5 Navigation and Obstacle Avoidance
- Laboratory
- New Laboratory
- 4.6 Robotic Demonstration
- 4.6.1 Natural Interactions
- 4.6.2 Requesting Information
- 4.6.3 Fetch-And-Carry Requests
- 4.6.4 Task Performance
- Discussions
- 5.1 The Speech Recognition
- 5.2. INTEGRATING SPEECH WITH CONTEXTUAL KNOWLEDGE
- 5.2 Integrating Speech with Contextual Knowledge
- 5.3. ADDITION OF SYMBOLIC GESTURES
- 5.3 Addition of Symbolic Gestures
- 5.4. COMPARING THE ITERATIVE AND SEQUENTIAL MULTIMODAL
- 5.4 Comparing the Iterative and Sequential Multimodal Ap-
- 5.5 The Robotic Demonstrations
- Future Work and Conclusions
- 6.1 Future Work
- 6.2 Conclusions
Add a Comment