COMPUTER ANIMATION AND VIRTUAL WORLDS

Comp. Anim. Virtual Worlds 2011; 22:229–237

Published online 12 April 2011 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/cav.397

SPECIAL ISSUE PAPER

Real-time mocap dance recognition for an interactive

dancing game
Liqun Deng1,2*, Howard Leung1, Naijie Gu2 and Yang Yang1,2
1
Department of Computer Science, City University of Hong Kong and USTC-CityU Joint Advanced Research Centre, Suzhou,
P.R. China
2
Department of Computer Science and Technology, University of Science and Technology of China and USTC-CityU Joint Advanced
Research Centre, Suzhou, P.R. China

ABSTRACT
In this paper, we present an interactive dancing game based on motion capture technology. We address the problem of real-
time recognition of the user’s live dance performance in order to determine the interactive motion to be rendered by a
virtual dance partner. The real-time recognition algorithm is based on a human body partition indexing scheme with
flexible matching to determine the end of a move as well as to detect unwanted motion. We show that the system can
recognize the live dance motions of users with good accuracy and render the interactive dance move of the virtual partner.
Copyright # 2011 John Wiley & Sons, Ltd.
KEYWORDS
interactive dancing game; motion capture; real-time motion recognition
*Correspondence
Liqun Deng, Computer Science and Technology, University of Science and Technology of China, Hefei, China.
E-mail: dlqun@mail.ustc.edu.cn

1. INTRODUCTION students to learn dance by themselves. Similar ideas have

Motion capture (mocap) has been prevalent in the
computer graphics industry because of the capability of
the motion data in recording expressive human motions in
the real world. It can produce vivid animations through the
virtual characters which help creating an immersive virtual
reality environment for entertainment, sport training, etc.
Performance-driven animation is an intuitive way for
applying the mocap technology which aims at reconstruct-
ing the live motions of users and mapping them into the
movement of the virtual characters. Animation authored by
the direct performance also allows the control of virtual
avatars or facilitates human–avatar interaction [1,2].
Computer puppetry [3] aims at directly mapping motions
of a real performer onto virtual characters. Magnenat-
Thalmann et al. [4] proposed to capture the teachers’ dance
motions using mocap and create a web 3D learning
platform that allows students watch the virtual avatar
performing the captured dance motions.
The captured motion can also be analyzed in motion-
based applications such as dance learning or sports game in
order to provide some feedback back to the user. Chan et al.
[5] implemented a dance training system that allows
been developed in the games such as Dance Central1 and
Dance Masters2 with the Kinect game console. Users are
required to mimic the dance animations, and their motions
are captured by a motion sensor and compared with the
animated motions to obtain the scores.
While performance driven animation and motion
mimicking games are entertaining, it is more desirable
to deliver intelligent reaction by the virtual avatar
according to the input of the user. This can enhance the
interactivity between the human user and the virtual
character so that the user feel more immersed in the virtual
environment. This requires a high level understanding of
the user’s input motion and is often achieved by
recognition techniques. In this paper, we consider this
type of application by designing an interactive dancing
game.
The overview of our interactive dance game is shown in
Figure 1. We assume that a dance sequence is connected by
a list of dance moves from a certain number of template
classes. We also assume that the mapping between each
1
http://en.wikipedia.org/wiki/Dance_Central.
2
http://wapedia.mobi/en/Dance_Masters.

Copyright ß 2011 John Wiley & Sons, Ltd. 229

Real-time mocap dance recognition
Figure 1. The overview of the system with block diagram.
dance move and its corresponding reactive move is
predefined. In the game, the dance performance of a user
is lively captured, preprocessed, and recognized in real
time by a novel online classifier. At the same time,
according to the recognition result, the suitable interactive
dance motion by the virtual dance partner is promptly
determined and rendered. This will give the user the
impression that he/she is dancing in collaboration with the
virtual partner. The primary contributions of this paper
include the followings:

We propose a novel approach to recognize dance
motions in real time. We develop an online human
motion recognizer based on a human body partition
scheme. We present our flexible matching method and
several rules which contribute to the good performance
on both continuous motion recognition and unwanted
motion detection.

We implement an interactive dancing game system that
can be used for dance training and entertainment. Exper-
iments and the user study demonstrated that our classi-
fier is effective in online motion recognition, and the
proposed system is well accepted by the investigated
users.
The rest of this paper is organized as follows: the next
section briefly reviews the related work. Next, the real-time
motion classifier is described and the interactive dancing
game application is illustrated in the following two sec-
tions, respectively. The performance evaluation section
presents the experimental results and user studies. The
conclusion and future work are provided in the final
section.
2. RELATED WORK
There have been a lot of efforts towards the offline analysis
of a single person’s motion. Kovar et al. [6] created a novel
concept of motion graph that constructed the mocap data as
a directed graph, which was demonstrated to be efficient in
230 Comp. Anim. Virtual Worlds 2011; 22:229–237 ß 2011 John Wiley & Sons, Ltd.
L. Deng et al.
generating different locomotion styles for animation. Lee
et al. [7] proposed to precompute the reactive motions for
avatars from a large collection of motion dataset so as to
reduce the time delay during the interactive control of
avatars in the animation.
Synchronizing the motion with the music is another
important aspect. Shiratori et al. [8] and Kim et al. [9]
identified the appropriate dance motions according to the
matching with the input music. Alankus et al. [10]
proposed an automatic approach to synthesize dance
motions with musical rhythms.
Lee et al. [11] worked on real time controlling the
avatars with mocap data. In their approach, the motion data
in database are preprocessed both by the Markov process in
lower layer and a clustering technique in higher layer.
Various classification techniques have been applied for
recognizing motions. Li et al. [12] applied singular value
decomposition (SVD) for feature extraction on motions
and proposed a new eigen feature based similarity measure
to classify them. Tormene et al. [13] proposed a new
variant of dynamic time warping named open-end DTW
(OE-DTW) which allows matching incomplete time series
patterns with complete ones. As another extension of
DTW, continuous dynamic programming (CDP) was also
utilized for human gesture recognition [14]. Liang et al. [2]
recognized the motions recorded by accelerometers using a
continuous hidden Markov model (HMM) based classifier.
In our prior work [15], we addressed the problem of
continuously recognizing the dance moves for a long dance
sequence, but did not consider the real time issue. The
above existing methods are not suitable for our case in
which the recognition decision is required each time when
several new frames are input instead of waiting for the end
of the entire pattern before starting the recognition process.
We need a fast method that does not require too much
training data.
Our recognition method is motivated by the success of
indexing techniques applied for motion retrieval [16–18],
which are efficient and fast in searching a large dataset for a
query motion. For example, Chiu et al. [16] proposed to
DOI: 10.1002/cav
L. Deng et al.
partition a human skeletal model into nine body parts and
construct an index map for each of the body parts through
self-organizing map (SOM) clustering. These maps are
then used for querying a motion in a long motion sequence.
Two advantages of the body partition based retrieval are:
(1) reducing the computation cost by partitioning whole-
body motions with high dimension into a set of body-part
motions of low dimension and (2) avoiding the disharmony
that usually occurs in dance between different body parts.
We use a similar scheme, and extend it from motion
retrieval to real-time motion recognition. In addition, each
time when a new block of input motion is recognized,
DTW is used to further extract its temporal alignment with
the corresponding template motion, and thus to decide the
exact interactive motion clip.
3. REAL-TIME MOTION
CLASSIFIER
The framework of our proposed classifier is shown in
Figure 1. It is divided into the indexing and recognition
stages, which are presented in the following subsections.
3.1. Indexing Stage
3.1.1. Motion Representation.
In this prototype system, we take Agogo dance into
account. We assume that this dance is composed of 19
classes of dance moves of different difficulty levels. Some
moves are symmetric in which the male and female
dancers’ motions are the same while in other moves, the
two dancers’ motions are completely different but
collaborative. More information about our dance moves
is described in the prior work [15]. For each class, 15 trials
are pre-captured by five subjects using the optical
system. The durations of the motion clips range from
85 to 360 frames. Each motion is measured by a set of 3D
rotations of 20 body joints (see Figure 2(a)).
Figure 2. (a) Human skeletal model. (b) Five body partitions of a skeletal model.
Comp. Anim. Virtual Worlds 2011; 22:229–237 ß 2011 John Wiley & Sons, Ltd.
DOI: 10.1002/cav
Real-time mocap dance recognition
On the other hand, to facilitate a higher-level description
of the motions, the joints of the skeletal model are grouped
into five partitions that include torso, left upper limb, right
upper limb, left lower limb, and right lower limb, as shown
in Figure 2(b). Hence each motion is represented by five
disjoint sub-motion matrices corresponding to the body
partitions along the frame time, which are clustered and
indexed separately.
3.1.2. Clustering Motions With SOM.
For each of the C classes, we select K trials as the training
source and build the index structure (In our system, C ¼ 19,
K ¼ 5). Thus, K  C sub motions for each body partition
exist. These sets of sub motions here are clustered using a
SOM based approach.
For the technical theory of SOM, we refer readers to the
book by Duda et al. [19]. In our case, for each set of sub-
motions specified by a certain body part, we collect all the
corresponding frames and cluster them using a two-step
procedure. The first step is to train a SOM with an existed
SOM toolbox [20], and take the weight vectors in the SOM
nodes as cluster centers. During SOM training, the initial
SOM parameters are set as the default of the toolbox [21].
After this step, we find that most of the resulted clusters are
not or rarely indexed by the training frames, thus the
second step is to iteratively refine the result. We first get rid
of the clusters that are not or rarely indexed (less than 40 in
our case), then resize the SOM map according to the
number of remaining cluster centers and retrain the frames.
This step is repeated until all the resulted cluster centers are
well indexed by the training frames. The rationale behind
this step is to reduce the noise in the result caused by
irregular training frames.
Figure 3 shows an example of projecting motions into
the clusters. By following the notation by Wu et al. [17],
the resulting sequences of cluster IDs are known as motion
strings and each motion trial is transformed into five
motion strings, with each frame represented by a vector of
231
Real-time mocap dance recognition L. Deng et al.

Figure 3. SOM clustering and projection.

five elements as the projections of the body parts. The final
sizes of the five SOM clusters are 10, 28, 33, 15, and 19,
respectively.
3.1.3. Building Indexing Maps.
This step aims to build five index maps corresponding to
torso, left upper limb, left lower limb, right upper limb, and
right lower limb, respectively.
First, for each motion class, the K trials are combined to
be trained as a template model (TM). Let L be the largest
lengths among the K trials. We scale the trials to contain L
frames by uniform scaling [22], and combine them (see
Figure 4 for example). Thus, the resulted model is also with
L frames, and each frame is composed by five items with
respect to the five body parts. Each item is a unification of
the corresponding K values. Second, the C TMs are used to
build the index maps. Each map is actually a hash table
consisted of two parts, entries and content. The number of
entries is exactly the number of the SOM centers of the
corresponding body part generated by the previous
subsection, and the entries are specified by the cluster
IDs (see Figure 5). The content of the entries is filled up by
traversing the TMs, that is, if the torso of the i-th frame of
model m contains ID j, then a new pair (m, i) will be added
into the content of the j-th entry of the torso map. The
average numbers of pairs for each entry in the five index
maps are 120, 102, 88, 217, and 139, respectively.
3.2. Recognition Stage
In this subsection, we apply the index maps for real-time
motion recognition. We propose a flexible matching
scheme to search the match of a query motion from the

Figure 4. An example of TM generation with two motion trials.

232 Comp. Anim. Virtual Worlds 2011; 22:229–237 ß 2011 John Wiley & Sons, Ltd.
DOI: 10.1002/cav
L. Deng et al.
Figure 5. An example of index map structure.
index maps, and the indexing scores are sequentially
accumulated. After determining the recognized class with
the largest score, DTW is used to extract the exact temporal
alignment between the input motion and the corresponding
recognized template motion.
3.2.1. Frame Matching Scheme.
Suppose TO, LU, LL, RU, RL are the indexed results from
the index maps of torso, left upper limb, left lower limb,
right upper limb, and right lower limb, respectively.
We define two matching schemes to calculate index score.
Definition 1. Whole Body Match (WBM)
If the same pair (TM #, Frame #) ¼ (m, i) exists in all five
match results, then it indicates that the input frame contains
very similar posture as frame i of the template model m,
and we claim that the input frame exactly matches frame i
of model m. The whole body match (WBM) can be
obtained by the intersection of the match results from each
of the five body partitions as indicated in Equation (1).
WholeBodyMatch ¼ TO\LU\LL\RU\RL
Definition 2. Left or Right Body Match (LRBM)
We find that many of the frames can hardly find exact
matches of all the five body partitions due to the spatial
variation between the query motions and the TMs. To
enhance the flexibility, we define an approximate match as
indicated in Equation (2). Namely, when the same pair (TM
#, Frame #) exists in the torso and left limbs or the torso and
right limbs, we consider it as an approximate match.
LeftRightBodyMatch

\ \ [
\ \ 
¼ TO LU LL TO RU RL
Figure 6. (a) Squat Down, (b) Stand Up.
Comp. Anim. Virtual Worlds 2011; 22:229–237 ß 2011 John Wiley & Sons, Ltd.
DOI: 10.1002/cav
Real-time mocap dance recognition
From the definitions, it can be observed that the WBM is
a subset of the LRBM because if all the five body partitions
are matched, this implies that the left body part and the right
body part are matched. The converse is, however, not true.
3.2.2. Sequential Accumulation.
The temporal order of motions is critical for the
recognition. Take two motions ‘‘squat down’’ and ‘‘stand
up’’ as shown in Figure 6 for example. These two motions
may lead to the same result since they share the same set of
postures but in reverse order. It is thus necessary to
accumulate the recognized results of the input frames to
ensure a strong sequential correlation.
To address the problem, we define two rules to match
the input motion with the template modes during
recognition. (1) Minimize the temporal difference. For
an input frame, there may be several matched frames
from the same template model tm, for example, (tm, f1),
ðtm; f2 Þ; :::; ðtm; fn Þ. In this case, if the match of the
previous frame is (tm, f0 ), then the pair (tm, fi) with which fi
is the smallest frame number among the matched frames
with fi > f 0 is chosen as the matched result from tm for the
current input frame. (2) Approximate match first and then
exact match for the body. To allow a more robust matching
and alignment of the input motion, for each input frame, we
first obtain the set of LRBM and determine the match by
rule (1), and obtain the set of exact match pairs under
WBM. If the approximate match pair is also an exact match
pair, then we increase the vote for the associate template
with e (e > 1, in our case, e ¼ 1.5), otherwise with 1.
3.2.3. Detecting End Point and Identifying
Unwanted Motions.
In practice, users usually perform dance moves continu-
(1) ously without any breakpoints. Also, it is common that
users may perform a move that is not defined by the system.
Thus, it requires the system to automatically both detect the
end points of the dance moves and identify the unwanted
moves from the online dance.
Our solution is based on two observations: (1) although
the embedded motions vary in durations, there exists a
minimal length for a valid move. If the duration of a motion
is less than this minimal length, then it is considered as an
unwanted move; (2), after the end of a move, the matched
result for the corresponding TM will drop sharply.
Assuming that TMi has the largest number of body
matches among the TMs, then if the current input frames
(2)
are not matched well with TMi for k times consecutively, it
233
Real-time mocap dance recognition
is considered as reaching the end point. If the length of the
current recognized motion is larger than a fixed length
minLen, it is considered as a meaningful dance move,
otherwise it is considered as an unwanted move. (In our
system, k ¼ 2, minLen ¼ 60).
After the end point of a recognized move is detected, the
input motion sequence is segmented, and the counts for all
the TMs are reset to zero, hence re-starting the recognition
process.
3.2.4. Extracting Temporal Alignment.
After the previous step, the label of the input motion is
determined. However, it does not guarantee that the
alignment between the input motion and the recognized
template motion is exact, which is important for the system
to produce smooth interactive motions.
We employ OE-DTW [13] to process the motions to be
aligned. One advantage of OE-DTW over traditional DTW
is that OE-DTW does not require the prior knowledge on
the length of the motion to be matched, and its distance is
determined by the last column of the distance table thus it is
suitable for online application. Suppose Q is an online
motion clip, R is a template motion, and Figure 7 shows an
example on OE-DTW. The current input Q matches the
suffix R1:::I of R.
For each move class, among the K trials, the trial with
minimal average DTW distance is chosen as the template
motion of this class. The real time input frames are
continuously aligned with the template motions and the
Figure 7. An example of OE-DTW alignment.
234 Comp. Anim. Virtual Worlds 2011; 22:229–237 ß 2011 John Wiley & Sons, Ltd.
L. Deng et al.
result hence is used to decide the exact interactive motion
clip to animate avatar in real time.
4. INTERACTIVE DANCING GAME
We have implemented the interactive dancing game
system. In the application, two modes, i.e., the training
mode and the freestyle mode have been taken into account.
The training mode aims to demonstrate pre-captured
dance motions such that users can watch and learn how
they should dance with the virtual partner. An exemplar
scenario is shown in Figure 8(a).
In the freestyle mode, the user is allowed to dance freely
in a given time period and his/her dance is lively captured
and processed by the game system in real time (see
Figure 8(b)). The music related to the dance is also played.
At the same time, the virtual partner is performing the
interactive motions to let the user feel more immersed in
the virtual environment. At the top part of the screen as
shown in Figure 8(b), several messages are displayed. The
message ‘‘interactive move: 4 (25%)’’ means that the
current input motion is recognized as the motion class 4,
and the percent completion of this move by the user is 25%.
For each run, we allow the user to dance for 20 seconds,
and the remaining time is shown on the screen. When a
meaningful dance move is detected, one mark is awarded to
the user.
5. PERFORMANCE EVALUATION
We examine the proposed classifier on both isolated and
continuous motion recognition, and compare the results
with existing methods. A user study is also conducted to
test the interactive dancing game.
5.1. Motion Recognition
As mentioned in the previous section, the database contains
285 motion trials, and we use one third of them for training.
Hence, we will test our classifier on the remaining motion
trials.
Figure 9 shows the recognition accuracy on the isolated
partial motions. Partial motions are quantified by a
parameter P that specifies the percentage of the frames
of each isolated motion that are fed into the recognizer. For
example, for P ¼ 10%, only the first 10% of the frames
from each isolated motion are used as input for recognition.
For P ¼ 100%, the whole frames of each isolated motion
are used for recognition. We can see that the accuracy
increases with P, and especially, when P ¼ 100%, the
accuracy reaches 97.89%.
To examine the performance on continuous motion
recognition, we also collected 100 test motion sequences
where each sequence consists of an average of five move
patterns embedded at various time instances. The
recognition result is provided in Table 1, which also
DOI: 10.1002/cav
L. Deng et al. Real-time mocap dance recognition

Figure 8. (a) Interactive dance game in training mode, where the right avatar is dancing the input moves, and the left avatar dancing the
interactive moves. (b) A snap of the scenario in freestyle model. The shot in the right bottom shows the real time dancing of a real user.

5.2. User Study

To test the interactive dancing game application, we invited

Figure 9. Recognition accuracy on isolated partial motions.
shows the result from CDP [14] for comparison. It can be
seen that for all three kinds of errors encountered in
continuous motion recognition, our method is more robust
than CDP.
10 subjects to try out the dance game. Among the subjects,
four of them have prior knowledge about Agogo dance,
while the others do not. Before trying this system, we first
briefly introduced this game including the mocap system to
them, and let them know what they should pay their
attention to when they are experiencing it. Especially, for
the ones who had no prior experience of Agogo dance, we
recommended them to watch some Agogo videos and take
the training mode before freely dancing in front of the
system.
After experiencing with our interactive dancing game,
the subjects were asked to give their feedback by a
questionnaire. Table 2 shows the questions that we

Table 1. Performance comparison for continuous motion recognition.

Our proposed method CDP

Number of insertion errors 4 12

Number of deletion errors 13 31
Number of substitution errors 34 52
Number of correctly recognized moves 453 417
Detection ratio (%) 90.60% 83.40%
Reliability (%) 89.88% 81.45%

Table 2. Questions for questionnaire and result.

No. Question Average mark

1 Is the game fun? 4.7 ( p < 0.001)

2 Is the game environment immersive? 4.6 ( p < 0.001)
3 Has the interactive motion been performed smoothly? 4.5 ( p < 0.001)
4 Is the delay of the interactive motions acceptable? 4.5 ( p < 0.001)
5 Are your dances correctly detected as you desire? 4.5 ( p < 0.001)
6 Is it effective for you to learn Agogo dance? 4.8 ( p < 0.001)
7 Is the training mode helpful? 4.3 ( p < 0.001)
8 Is the interface of this game satisfactory? 4.2 ( p < 0.001)

Comp. Anim. Virtual Worlds 2011; 22:229–237 ß 2011 John Wiley & Sons, Ltd. 235
DOI: 10.1002/cav
Real-time mocap dance recognition
provided. The subjects were required to give their marks
using the five-level Likert scale (1 means strongly disagree
and 5 means strongly agree) for each of the questions. The
last column of Table 2 shows the average mark as well as
the statistical significance of each question. Since high
mark represents positive feedback and vice versa, we can
see that our system impresses the subject greatly. The
marks from question 3, 4, and 5 support that the live dances
are accurately recognized, and that of question 6, 7, and 8
suggest that the system is well designed and nicely
interacts with users. The marks from question 1 and 2, on
the whole, indicate that the system can achieve good
performance and impress users.
6. CONCLUSIONS AND FUTURE
WORK
This paper presented an interactive dancing game based on
the mocap technology. We proposed a novel approach to
handle the real-time recognition of the user’s dance motion
based on human body partition indexing. The matching
was flexible in identifying the end of a move and detecting
unwanted motion. Experiments showed that our proposed
method has good performance on both isolated and
continuous motion recognition and positive feedback from
the subjects was obtained from the user study. However,
our classifier is trained with the trials of all motion classes
as a whole. Hence if a new motion class is added to the
system, it needs to redo all the training procedure.
As future work, we will consider introducing more kinds
of dances into this system to increase the complexity and
variety. Also, since different users have different dance
styles and dance progresses when they use this system for
learning purpose, it is a promising work to make the system
personalized. We may combine this work with our prior
work [5] to produce more meaningful feedback to the user
regarding his/her dance performance.
ACKNOWLEDGEMENTS
The work described in this paper was fully supported by a
grant from the Research Grants Councils of the Hong Kong
Special Administration Region, China (Project No. CityU
1165/09E).
REFERENCES
1. Chai J, Hodgins JK. Performance animation from low-
dimensional control signals. ACM Transactions on
Graphics 2005; 24(3): 686–696.
2. Liang X, Li Q, Zhang X, Zhang S, Geng W. Perform-
ance driven motion choreographing with acceler-
ometers. Computer Animation and Virtual Worlds
2009; 20: 89–99.
236 Comp. Anim. Virtual Worlds 2011; 22:229–237 ß 2011 John Wiley & Sons, Ltd.
L. Deng et al.
3. Shin HJ, Lee J, Shin SY, Gleicher M. Computer
puppetry: an importance-based approach. ACM Trans-
action on Graphics 2001; 20(2): 67–94.
4. Magnenat-Thalmann N, Protopsaltou D, Kavakli E.
Learning how to dance using a web 3D platform. In
ICWL ’07: Lecture Notes in Computer Science, Vol.
4823, 2008; 1–12.
5. Chan JCP, Leung H, Tang JKT, Komura T. A virtual
reality dance training system using motion capture
technology. IEEE Transaction on Learning Technol-
ogies, 17 Aug. 2010. <http://doi.ieeecomputersociety.
org/10.1109/TLT.2010.27>
6. Kovar L, Gleicher M, Pighin F. Motion graphs. ACM
Transaction on Graphics 2002; 21(3): 473–482.
7. Lee J, Lee KH. Precomputing avatar behavior from
human motion data. In SCA ’04: Proceedings of the
2004 ACM SIGGRAPH/Eurographics symposium on
Computer animation, 2004; 79–87.
8. Shiratori T, Nakazawa A, Ikeuchi K. Dancing-to-
music character animation. Computer Graphics
Forum 2006; 25(3): 449–458.
9. Kim JW, Fouad H, Sibert JL, Hahn JK. Perceptually
motivated automatic dance motion generation for
music. Computer Animation and Virtual Worlds,
2009; 20(2–3): 184–375.
10. Alankus G, Bayazit AA, Bayazit OB. Automated
motion synthesis for dancing characters. Computer
Animation and Virtual Worlds 2005; 16(3–4): 259–
271.
11. Lee J, Chai J, Reitsma PS. Interactive control of
avatars animated with human motion data. ACM
Transaction on Graphics 2002; 21(3): 491–500.
12. Li C, Zheng SQ, Prabhakaran B. Segmentation and
recognition of motion streams by similarity search.
ACM Transaction on Multimedia Computing, Com-
munications, and Applications 2007; 3(3): Article 16.
13. Tormene P, Giorgino T, Quaglini S, Stefanelli M.
Matching incomplete time series with dynamic time
warping: an algorithm and an application to post-
stroke rehabilitation. Artificial Intelligence in Medi-
cine 2009; 45(1): 11–34.
14. Mori A, Uchida S, Kurazume R, Taniguchi R,
Hasegawa T, Sakoe H. Early recognition and predic-
tion of gestures. In ICPR ’06: Proceedings of Inter-
national Conference on Pattern Recognition, 2006;
560–563.
15. Deng LQ, Leung H, Gu NJ, Yang Y. Automated
recognition of sequential patterns in captured motion
streams. In WAIM ’10: Lecture Notes in Computer
Science, Vol. 6184, 2010; 250–261.
16. Chiu C, Chao S, Wu M, Yang S, Lin H. Content-based
retrieval for human motion data. Journal of Visual
Communication and Image Representation 2004; 15:
446–466.
17. Wu S, Wang Z, Xia S. Indexing and retrieval of human
motion data by a hierarchical tree. In VRST ’09:
Proceedings of the 16th ACM Symposium on Virtual
Reality Software and Technology, 2009; 207–214.
DOI: 10.1002/cav
L. Deng et al. Real-time mocap dance recognition

18. Deng Z, Gu Q, Li Q. Perceptually consistent example- Naijie Gu is currently a professor in

based human motion retrieval. In I3D ’09: Proceed- the Department of Computer
ings of the 2009 symposium on Interactive 3D Science and Technology at Univer-
graphics and games, 2009; 191–198. sity of Science and Technology of
19. Duda RO, Hart PE, Stork DG. Pattern Classification. China (USTC). His main research
John Wiley & Sons: New York, 2001. projects include Parallel Algorithm
20. Vesanto J, Himberg J, Alhoniemi E, Parkankangas J. Design and Analysis, Communica-
SOM toolbox for Matlab 5. Technical Report A57, tions in Parallel or Distributed Net-
2000. work, Software Optimization, etc.
21. SOM Toolbox: http://www.cis.hut.fi/somtoolbox/
documentation/somalg.shtml. Yang Yang is currently a PhD stu-
dent in the Department of Computer
22. Keogh E, Palpanas T, Zordan VB, Gunopulos D,
Science and Technology at Univer-
Cardle M. Indexing large human-motion databases. sity of Science and Technology of
In VLDB ’04: Proceedings of the Thirtieth inter- China (USTC). He is also enrolled
national conference on Very large data bases, 2004; by a joint PhD program offered by
780–791. City University of Hong Kong
(CityU) and USTC. His research
interests include human motion ana-
lysis, human computer interaction,
AUTHORS’ BIOGRAPHIES
and e-learning.
Liqun Deng is currently a PhD stu-
dent in the Department of Computer
Science and Technology at Univer-
sity of Science and Technology of
China (USTC). He is also enrolled
by a joint PhD program offered
by City University of Hong Kong
(CityU) and USTC. His research
interests include human motion
analysis, human computer interaction, and pattern
recognition.

Howard Leung is currently an Assis-

tant Professor in the Department of
Computer Science at City University
of Hong Kong. He received his
BEng degree in Electrical Engineer-
ing from McGill University, Canada,
in 1998, MSc and the PhD degrees in
Electrical and Computer Engineer-
ing from Carnegie Mellon Univer-
sity in 1999 and 2003, respectively. Howard’s current
research projects include 3D Human Motion Analysis,
Brain Informatics and Intelligent Tools for Chinese
Handwriting Education. Howard is currently a member
of the Multimedia Systems & Applications Technical
Committee (MSATC) which is a subsidiary organiza-
tion of the IEEE Circuits and Systems (IEEE-CAS)
Society.

Comp. Anim. Virtual Worlds 2011; 22:229–237 ß 2011 John Wiley & Sons, Ltd. 237
DOI: 10.1002/cav