Scribd Logo
Upload

Welcome to Scribd!

  • Rhythmic Motion Analysis Using MoCap and Music

    Uploaded by

    Derian Montés
    6 views6 pages

    Original Title

    Rhythmic Motion Analysis using MoCap and Music

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document

    Copyright:

    © All Rights Reserved
    Download as PDF or read online from Scribd
    Flag for inappropriate content
    6 views6 pages

    Rhythmic Motion Analysis Using MoCap and Music

    Uploaded by

    Derian Montés

    Copyright:

    © All Rights Reserved
    Download as PDF or read online from Scribd
    Flag for inappropriate content
    of 6
    5-2 Rhythmic Motion Analysis using Motion Capture and Musical Information ‘Takaaki Shiratori Dept. of information and Communication Engineering The University of Tokyo Tokyo, 153-8505, Japan siratori@evlilssutokyo.ae jp Abstract ‘The number of Japanese traditional dancers has been de creasing. Without performers, some dances willbe disap eared, because they can not be recorded by conventional Imedia, suchas paper Me bave proposed an archiving method Specifically for dancing pattern Our method Bas four main stages: I. Digitizing motions by motion capture systems. 2 “Analzing motions, 3, Sytheing motions 4. Reproducing the dance mvions by CGand humanoid robo. Inorder to f= fecsely record the moving patterns, motion primitives are ex tracted. Each motion primitive describes each baste mation However. most previous primitive exraction fais to provide ‘the hychm which resulsinunhyehmie synthesized mation. Dt ‘his paper we propose a new motion analysis method, hich Integrates music rythm into the motion primitives. Ow ex- ‘periment confirmed tha our motion anal ilded the mo- fon primes accordance to the musi tn 1 Introduction Japanese rational dances are personaly taught fom one pewsoa to another person. Due to the decreasing numberof ancers, many of them wil be soon disappeared. n oder to ‘conserve the traditlonal dances, we have developed a digital srchiving method specifically for dancing patter as shown in Figure 1 = i =i H} bai Figure 1. Project Overview Atsushi Nakazawa Cybermedia Center (Osaka University Osaka, 560-0043, Japan nakazawa@ime.cme.osaka-t.acjp Katsushi Ikeuchi Institute of Industrial Science ‘The University of Tokyo ‘Tokyo, 153-8505, Japan ki@evl.iis.ustokyo.ac.ip ‘A motion capture system is one ofthe most effective meth ‘ds fr digitizing human motions, and commonly used to gen- erate human motions in computer graphics. The system pro- ‘ides ony the low-level information( motion). Increasing the the numberof data i used 16 cope withthe lack of high-Level information( Eg. moving shyt). Further information is i> ‘erpoated from the data, Inepoltion can be done by “mo- ion graph 156) fier banks{2] et, Dances canbe considered asthe special kind of motion and roguired the special kindof motion analysis, Motion beats) was proposed to capture the ythmic motion. Inerfaced sys- tem propose in [8] made it cay 1 eit and symhesize the dance motions Sapanose traditional dances cannot be effectively analyzed by conventional mothod, because every dance maton Yep resented asa sequence of “primitive motions”, whose spans ae based ona rhythm an corespond to the rhythmic patter ‘anderiying the motion. Motion primitive is defined asthe Ba- sic motion. A complex movement canbe described by he se ‘quence of motion primitives. In Japanese trains dances, primitive motions are called Mar. Tn our previously proposed methos(7), motion primitives are exacted based onthe speed of hands and fect. How ‘ve, the results sometimes lose the esence ofa dance and ‘the syathesind motion is not always ia tune withthe musi, because some of Mai, the primitive motions, ae segmented nthispaper, music rythm included in our motion-analsis ‘method 9 create the motion in une with the musi ‘An overall of our proposed method is presented in Section 2. We detect the musi chythm by beat racking method(3]) in Section 3. Our segmentation method for extraction of mo- tion primitives is described in Section 4. Our method was then verified and compared with another method in Setion 5 The paper is concluded in Seton 6 2 Algorithm Overview ‘An overall of ur proposed method i show in Figure 2. First, speed scquence of hands and center of miss it cake culated and candidates for segmentation areextacted from speed sequence. Then, if there isthe candidate around the susie rhythm, cach speed sequence of hands and center of (0:7. 9987XID¥817 00 Cm HEE ‘mass is segmented atthe musi ythm. Finally, motion se- quence is segmented by logical operation of each result of ands and cote of mass Figure 2. Algorithm Overview 3. Beat Tracking Method We adopted the beat tracking method proposed in [3], hich is used for detecting ehythm in wester music 3.4. Musical Elements for Beat Tracking ‘There are thee kinds of musical elements for beat rack ing + Onset ies + Chord Changes + Percussion Patterns [Experiment in [3] showed tht the three masical elements "were suitable for wester musi, suchas pop rock, et, How ve, Japanese dance musi is diferent to western music in both chor and shyt. In this section, we fst evaluate and selest musial elements appropristed to apanese music, Then we describe the extracting formulation athe processing of ‘he desired musical element. au ‘One of the good candidates for beat tacking agent is on- set timers tie). In Japanese dance music, there are few ‘hythm changes, such as rt, accel, te. Therefore, onset times s very effective for beat wacking, Onset times 34.2. Chord Changes ‘The frequency characteristic of chord in western music i i ferent rom that in eastern music. In addon, Japanese dance music hardly changes chord. Therefore this musial element ‘snot efficent for racking bea in Japanese music. 34.3. Percussion Patterns ‘The pereussion patter s extracted ase on the spectrum pat fer of percussion insrument(E g. drum, et). However, peteusion insrumens are not necessary in Japanese dance ‘nusie. Furthermore, [5] recognized percussion pattems fom bass drums and snare drums. Both are not used in Japanese dance music. Hence, this musical slement i not important in Japanese dance musi rom the above evaluation, we conclude that dhe dhythm Jin Jpanese dance music can be effectively exacted by onset ‘ims. The ater wo musical elements are unnecessary 32. Onset Component Extraction ‘When a sound is produced, the spectral power corespond ing to the frequency ofthat sound will increase. Each onset ‘component is extracted as shown in Figure 3. Figure 3. Onset Component Extraction: Caleu- lating how much power increases from "Pre- vPow" The magnitude of increasing spectral power dt, f atime (for fequency i defined as max(o(,f),p(t+ 1 f)) — PrevPow ae f=} (minin(t, £201, 4)) 2 PrevPow), © (otherwise), where o PrevPow = max(n(, 1) p(t~1, f),n(¢—1,4+1)), @) and p(s the spectral power at time t and frequency J PrevPow considers the spectral power between f 1, and f+ Tat time t~ 1. Therefore, an error onset time i hot picked up when the musical frequency is deviated by noise, players skill ec [Next at the fequency bands a user selected, beat acking systm calculates D(t) defined a follows: x ° Frequency range is divided int 7 bands(0-125H2, 125- 2s0HHz, 280:S00Hz, 300-000Hz, Th-2kHe, 2k-aktiz and ‘more than 4kH2). ach band coresponds {9 one octave as ‘sensed by human 33 Estimation of Beat Start and Beat Interval Inthis section, we describe how we estimate beat start and beat interval First, atlo-corelation Funtion(r)) of D2) is ealeu- lated. Avto-corelation function indicates the orthogonality of D(t) and D(t+ +) andis defined as follows: m0 FL pwove+) “ ‘where Nis the mumber of the data, In 8(r), a peak appear at rane, D(t)is very similar to D(C} Tena) Ad, Foy beat interval Figure 4), Ontaatnvctn Figure 4. Beat-interval Estimation ‘Beatstart time(s ealeulted from bet interval Tins 88 follows: = wma), o @sism. 4 Motion Analysis Considering Muse Rhythm We hypothesize that he sequence of human motion con: sists of motion elements which we define as "piitive ro tions” The primitive mations ae the set of basie mations and this concept has 3 great advantage for human motions pro= ‘cxssng(archve eit, ee) Japanese traditional dance is the sequence of short dant Mai"), This sore dance is used asthe primitive mo- tion in Japanese ractional dance Inbetween he short dance, dancers stop moving. So iis possible to extrac the primitive ‘motion(shore dance) based othe stop point. The stp point ten appears at music rym which canbe detected by on Set components, Because the spans ofthe primitive motions are based on the music rythm and corespond tothe thy ‘ic pattem underlying the motion. “The overall analysis method for human dance mations is presented in this section, 4.1 Motion Elements for Analysis ‘Ouranalysis method is based onthe speed ofa performer's hands and center of mass{CM). In most dances including Japanese traiional dance, movement of hand elated wilt the whole body expression. Therefore, the speed ofthe hands is useful for extracting the stop motions; however, tis not ‘ulcien for primitive extraction as pesented in he follow ing 0 cases. 1 Case: The dancer loses sense of rhythm When a dancer mistep(according to the music shyt), he tends to comet his dance by length- cningshortening his steps. Such movements make the detection ofthe stop motions loose, since the speed of body pars are not low enough 1 be de tected as stop motion. Consequently, stop mation isnot effetvely extracted by only the speed ofthe hands + Case: Dances are varied by the preference and the sender of performers. = Japanese tational dances ae taght from one person to another persons without the proper record. Consequently even inthe same dances di ferent dancers performs eifeently. While he dif- ference among professional dancers snot visually distinc it has much effets on motion analysis. In addition to the motion of the hands, ou algorithm use ‘he motion of the body's CM. The motion of CM represents ‘he motion ofthe whole body; thus, the effets of misstep and individaldiference ae les. Primitives ean then be ecc- tively extracted. The calcslaton of the body's CM is derived fom Figure 5: Figure 5. Center of Mass, Fire(a) presents where the position ofthe center of mass ‘ofeach body paris and Figure (6) presents the percentage of ‘mass ofeach body pat 42. Segmentation of Dance Motions considering Music Rhythm ‘There are three stops segment the motion sequence 1. Speed Calculation 2, Segmentation Candidates Exttion 3. Segmentation Considering Music Rhythm ge 42.1 Speed Catealation ‘The captured motion dats is recorded in global coordinate system To eaeulate speed of the hands, we define the local coordinate system as follows, The origin ofthe local coadi- hate system is the mide of human Waist The thre ans of the local coordinate are shown in Figure 6. 2 axis is parallel to the wast, y axis is perpendicular to the wast and = axis ‘etal. Relative motion of individual body past the entire body 8, therefore, the mation in local evordinate system Figure 6. Body Center Coordinate System (On the other hand, the CM is caeulated inthe global eo- ordinate system, ‘The effect of nose is reduced by smoothing motion se- quence with gaussian ter before extracting candidates 2.2 Segmentation Candidates Extraction ‘After calculating speed, the systems extract the candidates for segmentation. The candidates are defined asthe local- ‘inimam point which satisfies the folowing two condi- tions Figure 7) 1. Bach candidate i a local minimum in speed sequence ‘and the local minimam less than the minimumspoed threshold 2. The local maximum between two suecessve candidates an the maximur-speed threshold. is larger Figure 7. Segmentation Candidates Extraction 42.3. Segmentation Considering Musie Rhythm [At each speed sequence, our method tests whether there tye candidates arOund music thythm detected by onset timesttoa)- IFthere isa candidate, ts possible that there ise stop point around ma andthe motion sequence is seg- tented a ine. Figure illstates the segmentation process Inthe igure, there are no segmentation candidates around Ist and Sed music rythm and the motion isnot segmented a these points. Because there are segmentation candidates round 2nd.ané sth musi hythm, the motions ae sepmented at these points shiver SS Figure 8. Segmentation considering Music Rhythm Logical operation i used to confirm whether fg. i the stop point. The operation cheeks the harmony ofboth hands andthe CM and is defined as (Resultof Left Hond 1 Result of Right Hand) Result of Center of Mase o AND operation is applied because the dancer stop moving catisely during the stop betveen primitives. OR operation is applied, o that she stop point undetected by hand motion is ‘compensately detected from the motion of CM S Results SL Experimental Data ‘Our proposed method was tested with tree dance dts male an female Asu-bandaisan, and ongara-bushi. These ‘motion data were captured by Vicon Motion Systems, whieh is optical motion capturing system and recoeds the position of 13 makers on a person, The sampling tate of Aizu bandaisan {ance and Jongare-bushi were 120 fps and 200fps, respec tively The msi was stored in wa format. The data size was 16 bits andthe sampling rate was set at 3200012 52 Results of Beat Tracking ‘To extract the onset components, frequency spectrum Was

    You might also like