P. 1
Speech Controllable 3D Animated Tai Chi Tutor in Augmented Reality

Speech Controllable 3D Animated Tai Chi Tutor in Augmented Reality

|Views: 52|Likes:
An essay for the 2011 Undergraduate Awards (Ireland) Competition by Frank Han.
An essay for the 2011 Undergraduate Awards (Ireland) Competition by Frank Han.

More info:

Published by: Undergraduate Awards on Aug 31, 2012
Copyright:Attribution Non-commercial


Read on Scribd mobile: iPhone, iPad and Android.
See more
See less


Speech Controllable 3D Animated Tai Chi Tutor in Augmented Reality

Undergraduate Award 2010

As the world’s entering its 21st century, people are paying more and more attention to their physical and mental health against the threats brought by modernization such as over anxiety, high pressure, lack of sports, etc. On the other hand, the people suffering from cardiovascular diseases which just appeared in the recent decades nearly double its number every five years around the world. People must take actions to improve their physical and mental fitness to enjoy a healthier and longer life. Long form Tai Chi is a 108 movement sequence designed to improve health in body and mind. Researchers have found that Tai Chi practice shows favorable effects on the promotion of balance control, flexibility, and cardiovascular fitness and reduced the risk of falls in both healthy elderly patients, and those recovering from chronic stroke, heart failure, high blood pressure, heart attacks, multiple sclerosis, Parkinson's, and Alzheimer's. Tai-Chi's gentle, low impact movements can burn more calories than surfing and nearly as many as downhill skiing. There have also been indications that Tai Chi might have some effect on noradrenalin and cortical production with an effect on mood and heart rate. The goal of the project we are talking about in this dissertation is improving both physical and mental fitness of people, especially the old group, by encouraging them to learn Tai Chi with a vivid 3D Tutor in an augmented reality game environment. The emphasis on human considerations and in particular speech as a mode of Human Computer Interaction (HCI) is also essential for a successful project implementation. Users will be able to see the 3D AR Tutor through a special pair of glasses or a video camera, and interact with him by speech. For more docs, presentations, videos about this project, please download the package from http://d1082573.u30.hosting.digiweb.ie/presentation.zip (Name and University Details have been removed from those files for ANONYMOUS consideration). This project has been awarded one of the best projects in the university and ranked the 2nd in Imagine Cup 2010 Ireland Final out of over 400 candidates.

Tai Chi, 3D Programming, XNA, Animation Components, Speech Control, Augmented Reality, Motion Capture, Human Computer Interaction (HCI), Goblin XNA, AR Tag, Microsoft Speech SDK, 3D Max Studio, Maya, Motion Builder, Gypsy 5 Motion Capture Suit.

1. Overview
This project explores the possibilities and approaches of teaching Tai Chi in Augmented Reality environment by building a 3D vivid Tai Chi tutor. Here are two screenshots of the application.

Figure 1: Desktop Application

Categories and Subject Descriptors
[Computer Science and Information Technology]: Multimedia Information Systems – Animation, Artificial, Augmented and Virtual Realities

General Terms
Speech Control, 3D Programming, Design, Reliability, Human Computer Interaction (HCI) Augmented Figure 2: Augmented Reality Application 1

2. Challenges We Are Facing
Naturally, health becomes the most popular daily topic of people coming from all cross-section of society. From the boom of the health service industry and the higher consume of the health check, we can find that people are becoming more and more sensitive on their health. On the other hand, according to the rent survey by Medicine Net, one of the eight adults suffers from stress or anxiety. [1] Cardiovascular diseases (CVDs) are increasing significantly and becoming No. 1 human killer. By 2030, almost 23.6 million people will die from CVDs, mainly from heart disease and stroke. [2][3] Obesity is another big cause of death. The average suffering from it is becoming younger and younger. Fortunately, that is preventable at early stage. The largest desirable feature of Tai Chi is that it really does well to human health. Since the early twentieth century, when Tai Chi was widely spread for the first time, medical studies of Tai Chi have found that Tai Chi practice shows some favorable effects on the promotion of balance control, flexibility, cardiovascular fitness, and reduced the risk of falls in both healthy elderly patients, and those recovering from chronic stroke, heart failure, high blood pressure, heart attacks, multiple sclerosis, Parkinson’s, and Alzheimer’s. Tai Chi’s gentle, low impact movements can burn more calories than surfing and nearly as many as downhill skiing. Also, focusing the mind solely on the movements of the form purportedly helps to bring about a state of mental calm and clarity, some indications shows that Tai Chi might have some effect on noradrenalin and cortical production with an effect on mood and heart rate. Through Tai Chi’s use of diaphragmatic breathing and focused meditation, this form of exercise has a relaxing effect on the individual. There has been scientific evidence showing that practitioners show improvements in selfesteem, self-confidence, sleep and mental health as the result of participating in a Tai Chi program. [4] Besides, Tai Chi is a martial art, the physical techniques of Tai Chi are described in the Tai Chi classics as being characterized by the use of leverage through the joints based on coordination in relaxation, rather than muscular tension, in order to neutralize or initiate attacks. The slow, repetitive work involved in the process of learning how that leverage is generated gently and measurably increases, opens the internal circulation. In September 2009 the National Institutes of Health held its first ever Mind-Body Week, Bill Douglas, the founder of World Tai Chi & Qigong Day, spoke on the usefulness of Tai Chi as part of modern healthcare.[5]

Figure 3: CVD - No. 1 Killer

Figure 4: CVDs are Increasing Significantly

3. Introduction to Tai Chi
Figure 5: Tai Chi Movements About 4,000 years ago, ancient Chinese wise men uncovered the secret of health. They called it Chi. Over centuries, a technique has been developed to replenish people’s Chi, which is Tai Chi. Today, millions people practice Tai Chi daily in order to live a more balanced and harmonious life. 2


Project Development and Implementation

Here, we are going to talk about the development and implementation of this project.


Model Building

The job can be divided into two parts, the animation editing and 3D model building. We used Motion Builder 2010 to edit the recorded the animation data to make sure it work smoothly as expected and 3ds Max to build a Panda tutor model and applied the skeleton animation into it.

Figure 6:System Architecture

4.1 Motion Capture
In order to make the Tai Chi tutor, the first thing we need to do is to get the tai chi movements sequence data. We used the Gypsy 5 Motion Capture Suit in a research lab to capture the skeleton animation data of Dr. Bryan Duggan who has practiced Tai Chi for over 15 years.

Figure 8: 3D Model Building

Figure 7: Motion Capture



Speech Control

Our project offers a speech recognition system compatible with Windows 7 platform. Windows 7 comes with a new feature named speech recognition engine. Speech recognition applications have a significant improved compared applications in the past operation system. The recognition is more accurate, more reliable than the past. Windows 7 Speech Control SDK is an Application Programming Interface (API) provided and developed by Microsoft providing users direct access to the Speech Recognition Engine and Speech Synthesis for developing speech based applications running on Windows 7 OS. Figure 9: Motion Editing


Augmented Reality

Augmented Reality (AR) is virtual reality (VR) added to a real place, imagery or object. VR creates a simulation of reality, but AR does not. Instead, it takes a real object or space as the foundation and incorporates technologies that add contextual data. The purpose of AR is to create the sensation that virtual objects are present in the real world, to achieve this purpose; AR combines VR components with the real world. AR has been put to use in a number different of fields, including digital games and medical imaging. One of the simple examples is using a webcam (or camera on your mobile device or PDA) and computer monitor (or mobile device or PDA), you can see the display to view both real world and virtual object project on the marker pattern. You can move around the marker and rotate the marker, at the same time, you will see virtual object, model move or rotate as well. [6]

Figure 10: Model Building


3D Programming

Nowadays, most computers have very powerful graphics card which can store complex 3D objects in its memory and project it into a 2D screen. Our story begins here. In 3D world, we have 3 axis namely X, Y, Z to represent the world coordination system. We use a vector to represent the location of an object and a transformation matrix or quaternion to represent its orientation. There are 3 basic transformation approaches to manipulate the object, translation, rotation and scale. Translation is used to move the position of the object. Rotation changes the orientation of it. Scale increases and decreases its size. Microsoft XNA 3.1 framework is an excellent toolkit for developing application in 3D space. And we used an open source library called “Animation Component Library” to load and control the animations.

Figure 11: Augmented Reality

5. Test and Evaluation
Test is definitely a key part of this project. Basically we did two groups of testing, unit test and usability test. The feedback from the tests really helped us make several significant improvements of this application and evaluate the our research goals of the possibilities and approaches of teaching Tai Chi. 4

Table 3: Unit Test Code Sample

5.1 Unit Test
Units are the smallest building blocks of software. In object oriented language, individual classes or functions make up units, and in structure language, modules such as functions, process make up units. The term unit testing refers to individual testing of separate units of the software system such as methods and classes. The Following case is one example to do the unit test. [7] Table 1: Unit Test Scenario Sample Function Description Set Scale The Set Scale function of this software is to allow end user to change the scale for the panda model. Analyze that the function is according to requirements, and also check for the existence of defects or errors in the software. Table 2: Unit Test Plan Operation performed Set different scale value for the panda model Boundary tested The scale of panda must in the range of 1 to 2, and allows floating value. Actual result Test passed

[Test] public void ScaleTesting() { // Scale control module for (int i = 0; i < 1000;i++) this.SetScale(i); } public void SetScale(float scaleParam) { scaleParamX += scaleParam; scaleParamY += scaleParam;

Testing purpose

scaleParamZ += scaleParam;

_sdt = System.DateTime.Now; _isDisplay = true;

_strDisplay = "Scale " + (Convert.ToInt32((scaleParamX - 1) * 100.0) + "%"); Assert.IsTrue(MathHelper.Clamp((float)(scaleParamX) , 1.0f, 2.0f) <= 2.0 && MathHelper.Clamp((float)(scaleParamX), 1.0f, 2.0f) >= 1.0); }

Figure 12: Unit Test Result Sample

5.2 Usability Test
5.2.1 Problem Statement and Test Objectives


This document describes a test plan for conducting a usability test during the development of Tai Chi Tutor. The goals of usability testing include establishing a baseline of user performance, establishing and validating user performance measures, and identifying potential design concerns to be addressed in order to improve the efficiency and end-user satisfaction. The usability test objectives are: •Exercise the Tai Chi Tutor under controlled test conditions with representative users. Data will be used to access whether usability goals regarding an effective, efficient, and well-received user interface have been achieved. •Establish baseline user performance and user-satisfaction levels of the user interface for future usability evaluations.

Sr. Mary Flanagan runs a free Tai Chi class in Aungier street, Dublin. We have regular contact with her and we would like to provide the people attending the class an opportunity to play around with the application and try to accomplish the list of tasks. Doctors
Prof. Arthur Tanner is a surgeon doctor in Royal Surgeon Institute. He is just entering his 70s and has interests in learning Tai Chi. We are glad to have him try around with the application and give us some feedback from a professional doctor point of view.

5.2.3 Test Length
It is estimated that the usability test will take each participant approximately 25 minutes to complete. The following is an estimate of the amount of time each participant will spend performing the different phases of the usability test. • • • Briefing: 5 minutes Instructions: 15 minutes Debriefing: 5 minutes

5.2.2 Potential Users
The participants' responsibilities will be to attempt to complete a set of representative task scenarios presented to them in as efficient and timely a manner as possible, and to provide feedback regarding the usability and acceptability of the user interface. The participants will be directed to provide honest opinions regarding the usability of the application, and to participate in pre/postsession subjective questionnaires and debriefing.

5.2.4 Enable Ireland
Enable Ireland provides free services to over 3,500 children and adults with disabilities and their families from 15 locations and in local communities countrywide. Covering childhood to adulthood our expert teams work with the individual and their family on a plan for each life stage.


The Usability Test Report will be provided at the conclusion of the usability test. It will consist of a report and/or a presentation of the results; summary, test objectives, description of method, data analysis, presentation of results and a list of recommendation.

5.3 Evaluation and Conclusion
The Unit test results are positive to us. Most of the test cases passed and we found out several bugs as well. For example, at the end of a music play, the next music index was not returned correctly. A small number of those kinds of bugs have been fixed. During the usability test, most of users were impressed by the application at the first glance. We analyzed the report for the usability and made the following improvements based their feedback. We also know that teaching Tai Chi in Augmented Reality is a very good approach to encourage people to practice, in order improve their mental and physical health and live a more balanced and harmonious life.

Figure 13: Usability Test with Enable Ireland

Firstly, I would like to thank my supervisor Dr. Bryan Duggan who gave me a lot of encouragement and help. Tai Chi Class in Aungier Street


We also want to acknowledge our gratefulness to Dr. James Condron for his generous help with Tai Chi movements capture by Gypsy 5 Motion Capture Suit in his lab. During the development and test phases, we got a lot of support and advice from several friends, Enable Ireland and University Tai Chi class run by Sr. Mary Flanagan. I would like to take every chance to give them our biggest thanks.

[3] Organization, W. H. (2009). Cardiovascular Diseases. World Health Organization. [4] Daily, H. (2008). Tai Chi Health and Fitness Benefits. Health Daily. [5] Tai Chi Health and Fitness Benefits, Bestinmartialarts, 2008 [6] Ohan Oda, “Goblin XNA Guide” September 25, 2009. [7] Hong Zhu, Patrick A. V. Hall, John H. R. May “Software unit test coverage and adequacy”, ACM Computing Surveys (CSUR) 1997, ISSN:0360-0300

[1] Net, M. (2009). Generalized Anxiety Disorder (GAD). Medicine Net. [2] Cardiovascular Disease - body, history, cause, rate, person, human, Risk Factors, Treatments, Reoccurrence


You're Reading a Free Preview

/*********** DO NOT ALTER ANYTHING BELOW THIS LINE ! ************/ var s_code=s.t();if(s_code)document.write(s_code)//-->