You are on page 1of 8

P E RVA S I V E C O M P U T I N G I N S P O R T S

A Personalized Music
System for Motivation
in Sport Performance
The IM4Sports music system helps exercisers select music that suits their
training programs, reflects and guides sport performance, and collects
data for adapting training programs and music selections.

M

usic can have a positive effect
on enjoyment of and motivation for performing physical
exercise, especially for recreational exercisers who work
out at submaximal intensity for health purposes.
The needs of this user group in terms of enjoyment and motivation are particularly high and
the potential positive effect can be significant (see
the “Solo Exercise and Music”
sidebar). The happy marriage
Gertjan Wijnalda, Steffen Pauws,
between solo endurance sports
and Fabio Vignoli
and music has already resulted
Philips Research Eindhoven
in the development of conHeiner Stuckenschmidt
sumer products. For example,
Vrije Universiteit Amsterdam
the Philips-Nike MP3Run acts
as a portable music player and
a performance and training
display and logging system (www.nike-philips).
In available products, however, music selection
and playback aren’t coupled to the exerciser’s
performance.
We developed a personalized music system
called IM4Sports (Interactive Music for Sports)
for individual exercising, although running is the
prime target. As figure 1 shows, our research prototype of the system consists of a personal computer, a portable music flash player, a heart sensor strap, and a pedometer.

Installation
First-time use of the IM4Sports system
requires an installation stage to personalize the
10

PERVASIVE computing

system using a personal computer. In a dialogue
with the user, the system requests relevant personal data (such as name, gender, and age), exercise and physiological data (such as weight, resting heart rate, and maximal or peak heart rate),
and the level of sports experience (such as beginner or experienced). Users also have the opportunity to alter or add exercise definitions and to
indicate their likes and their dislikes of music in
terms of songs, artists, or genres with respect to
the exercises.
In addition, users must analyze and annotate
their personal music collections (for example, in
MP3 format) by attaching attribute information
to each individual song. These attributes reflect
tags of catalog data or musically intrinsic features
such as unique identification, title information,
artist name, genre, duration, file size, and tempo.
Online services from third parties or audio feature extraction algorithms for computing tempo
can provide values for these tags.1,2 Multiple users
can use the system, although each has an individual profile. Users can always come back to the
installation stage to reset their data or change
their music collection.

Preparation stage: Before exercising
We define the usage of the system in three
stages: preparation, exercise, and feedback (see
figure 2). In the preparation stage, the user can
select a training program and suitable music using
a personal computer. (For more information
about training programs, see the “Using a Train-

Published by the IEEE CS and IEEE ComSoc ■ 1536-1268/05/$20.00 © 2005 IEEE

and Fast-Paced Classical Music on Progressive Cycling to Voluntary Physical Exhaustion. pace. 39. Small. and tempo. ing Program” sidebar. Marisi. “Effect of Music and Rhythm on Physical Performance. The IM4Sports system consists of a portable player with in-ear phones. For instance. 109–113. intensity. Constraints can also reflect the user’s music preferences during the exercise or restrict music attribute values to suit the exercise. “The Effect of Music Type on Running Perseverance and Coping with Effort Sensations.” Psychology of Sport and Exercise. Leigh. 1997. vol. In other words. Sport Behavior. 89–109. and P.” J. vol. A.4 At approximately 70 percent MHRR and higher. A.C. pp. Training programs are available for use or adaptation. Szabo. exercisers are no longer able to dissociate themselves from the incoming painful sensations. developing a positive mood.) The user downloads the music. music selection copes with storage limitations on present-day portable flashbased players. they must directly fight against the pain to persevere.3 Also. 2. its attribute information. To this end. pain. pp. 220–225. and M. Users select their preferred music from a larger collection that fits the training program to ensure a hassle-free sports and music experience. doubling the pace of the music at 70 percent of MHRR enables exercisers to postpone their time to fatigue. For instance.” J. 3. 4. it will enhance their performance. vol. and a personal computer. vol. it doesn’t consider performance but requires the exerciser’s full concentration.Solo Exercise and Music M any individual endurance sport practices require motivation for the exerciser to persevere. if exercisers are explicitly instructed to keep pace with music playing. 49. Different training exercises have different characteristics in motion. it seems that the change in tempo helps them in prolonging their period of using a dissociative coping strategy. athletes employ dissociative and associative strategies. In addition. In particular. To cope with the physical demands of their exertions.1. “The Effects of Slow. G. compared to the absence of music. no. In contrast to hard drives. artist.1 REFERENCES 1. and fatigue. Music won’t help them sustain their effort.2 If the physical load doesn’t surpass the submaximal level—that is. the system generates a play set (essentially an unordered set of songs) for each exercise. although it can often bring greater enjoyment to the task. Karageorghis. M. or doing some mental arithmetic. 2. Terry. flash memory doesn’t have moving parts. pp. Sports Medicine and Physical Fitness. heart rate sensor belt. Figure 1. endurance was enhanced in a stationary constant-pace cycling task only when exercisers synchronized their pedaling rate with music. no. “The Psychophysical Effects of Music in Sport and Exercise: A Review.I. Music can also help an exerciser to dissociate from low to moderate exertive feelings and eventually result in performance improvement. pedometer using acceleration sensors. they can dissociate from the physical demands by producing a train of thought. Anshel and D. no. Instead. PERVASIVE computing 11 . 1999. 2004. no. different exercise characteristics lead to different music selections in terms of genre. 2. JULY–SEPTEMBER 2005 warming up is different from running. if it’s below 70 percent of maximal heart rate reserve (MHRR)—they’re able to voluntarily divert their attention from internal sensations. 3. 1. or the user can create one from scratch. especially when these exertions evoke feelings of boredom. Constraints declare what songs should and shouldn’t be included in a play set. for which they require high levels of motivation and exertion tolerance. Obviously. C. Tenenbaum et al. so it can withstand movement during physical activity. and the training program to a portable player to take along.. pp. 54–68. Generating play sets for a training program then comes down to selecting songs from the personal music collection that satisfy all constraints.” The Research Quarterly. 20. 1978. 5. and duration.

Intensity is expressed as a percentage of maximal heart rate reserve (MHRR). This constraint is satisfied if all the selected songs’ tempos fall within a predefined lower and upper bound. • the exercise intensity. The system sets the lower and upper bound on the basis of what we know about the user’s running performance and the exercise intensity and stringency. sic Preparation stage Music selection User characteristics Feedback stage Mu Excersizing stage Physiological data & user feedback Performance data Off-line stages If we know that the user prefers slow music for a particular exercise. this percentage specifies a desired heart rate zone. the training program is part of a complete coaching plan to achieve a well-defined long-term goal such as losing weight. In this way. the user likes music from particular genres or artists. which specifies at what level of exertion the exerciser should perform the exercise. As table 1 shows. which tells how Run strictly or loosely the exerciser should Recover perform the exercise in terms of duration Stretch and intensity. For instance. Duration Intensity Stringency 20 minutes — Loose 5 minutes — — 20 minutes 70–80% MHRR* Strict — 50% MHRR* Loose 5 minutes — — 3 km — — Variable — — *Maximal heart rate reserve. we can instantiate a range constraint. and music playback. Jog For particular exercise types.P E RVA S I V E C O M P U T I N G I N S P O R T S Figure 2. on the other hand. coaches. music selection. The different usage stages in integrating a sport training program. for instance. If. about 60 percent rock music Using a Training Program M any recreational exercisers and elite endurance athletes work with training programs. If the user dislikes a song or music from particular genres or artists while exercising. or if we can infer from the expected stride frequency in running what musical tempos we need to include. the aerobic heart rate zone for improved Type endurance and optimal cardiovascular training is at 70–80 percent of MHRR. How long it will take to recover from an exertion is also unknown beforehand. Note that a training program can also consist of a single exercise such as a run of one hour for those who prefer a loose run across the park instead of committing themselves to training programs. users can state that they want. the definition of an exercise includes • the type of exercise. possibly at a prescribed intensity level. which prescribe the sequence of physical exercises scheduled in a single training session. working towards peak performance. . which specifies the period of time that the exercise should take. which refers to an exertion within a sport. Typically. For 12 Cool-down instance. it’s presumably close to rest level. Sports physiologists. heart rate during warm-up or stretching is less relevant. parameters are irrelevant or unpredictable. Warm-up and Stretch • the exercise stringency. or the runners themselves can create plans and training programs. the system instantiates a counting constraint that restricts and forces the number of occurrences of genres or artists in the play set. TABLE 1 A sample training program. or extending endurance. the system instantiates an exclude constraint that sees that the play set won’t contain music with these attribute values. • the exercise duration. such as running.

we accept the new one and enter the local search’s next iteration. It steps from solution to solution by applying a random. Besides standard linear playback. the search process is cooling down. 256/512 MB). We assume that the music’s tempo is reasonably stable for large parts of the song. the system adapts the music playback to the user’s performance or training goal (for example. Especially for these problems. The user then only needs to download the selected music to the portable device. music plays on the portable player to either support or guide user performance as expressed in heart rate. the local search considers complete solutions or play sets. stride frequency. The tempo speeds up or slows down to motivate the user to speed up or slow down by synchronizing his or her steps with the music. instead of striving for an exact solution. If the new solution is worse.4 Simulated annealing requires a predefined cooling schedule for temperature control. The system continually adapts the tempo to the user’s stride frequency to support running at varying stride frequencies without the user having to concentrate on keeping time. When the user is exercising. which makes sure that all the songs the user ultimately downloads to the device are different. Depending on the required motivation. In addition. • Pace-matching mode. small change at each iteration such as adding a song. we’ve developed the following advanced music playback modes: Exercising stage • Pace-fixing mode. no feasible solution exists. which decreases with the amount of deterioration and during the incoming data points) as the user’s heart rate. Running times are about five seconds on a standard PC platform to select music that sufficiently approximates a set of constraints. stride frequency for runners). course of the algorithm. the system needs to select a set of songs that has a similar total duration. In this case. We define stride frequency or pace as steps per minute (spm). It monitors and records user performance data such as heart rate and movement frequency (for example. The total penalty is defined as a weighted combination of all penalty functions. deleting a song. This mode aims to help the runner evenly distribute effort over a distance and somewhat resembles cruise control in cars. The exercises in the training program dictate the song order. known as simulated annealing. the mean of the latest five PERVASIVE computing 13 . Satisfying a set of constraints is a hard combinatorial problem to solve. Although a stride comprises Depending on the required motivation. A sensor attached to a chest belt with a wireless transmitter and a receiver for the player measures heart rate. if all music should fit on a portable player. This constraint is satisfied if the system selects songs with a sum of durations (or file sizes) that falls within a predefined minimum and maximum value.3 If constraints conflict. the goal is to arrive at a performance level as defined in the training program.and 40 percent popular music while exercising. For instance. The sensor transmits a pulse for every heart beat. we equate it to one step for practical reasons. Analogously. A pedometer measures stride frequency. Therefore. To find a play set that minimizes the total penalty.4 Our local search approach defines each penalty function to be zero if the system meets the constraint. we use a summation constraint. the user controls the playback and can skip or prolong songs and exercises using interactive controls on the player. The player uses a five-point moving average of incoming pulse intervals (that is. we compute an approximate solution by using a local search approach in which the constraints are translated into normalized. and speed or covered distance. to be larger than zero if it doesn’t. the player uses a five-point moving average for this as well. piecewisely linear penalty functions. and to increase with the amount of violation of the constraint. Weights indicate the severity of not meeting a constraint. the system needs to select a set of songs that doesn’t exceed the device’s maximal storage capacity (for example. including those already on the portable device and those used in previous training sessions. the system adapts the music playback to the user’s performance or training goal. or JULY–SEPTEMBER 2005 replacing one song with another. Obviously. two steps. we use an all-different constraint. If an exercise has a limited duration. Music plays at a constant tempo to motivate the user to synchronize his or her steps with the music for enhanced endurance. selecting more music than can fit on the portable player is more severe than leaving out a preferred artist’s song. we use a standard linear cooling schedule. In other words. If the newly generated play set is better than the original one. or the number of ground contacts the user’s feet make in one minute. reaching and staying within a certain heart rate zone). we might accept the solution with a certain probability. • Pace-influencing mode.

slower songs will adapt more swiftly than faster songs. the system adapts to the user too slowly.5 The scale factor represents the ratio between the desired tempo and the song’s original tempo. To allow the same amount of time for a maximally allowed tempo change for all songs. Propagate the tempo change to the tempo the determined subgoal desires. the system speeds up the tempo to 156 bpm. if the current song has a moderately fast tempo of 14 PERVASIVE computing the system selects a new song to replace the current one on the condition that the song has been playing long enough (say. The system selects other songs if it can no longer match the tempo to the user’s stride frequency. In this case. the system can’t find a valid match inside the song stretch range. this results in a stretch range of 85–125 bpm. The new song starts to play at 156 bpm. 3. a change from 100 to 120 bpm would take 100 − 120 ¥ Tm = 250 tm( 100Æ 120) = 85 − 125 msecs. Determine the heart rate goal and. The system uses a linear function to calculate the actual time to change from a given tempo to a new one. the stride frequency and tempo subgoals. This function. this needs to 125 bpm (with a stretch range of 107–156) and the user walks at a stride frequency of 117 spm. When a change in tempo occurs instantly. On the other hand. Both the original song and its playout tempo are expressed in beats per minute (bpm). the system would slow down the tempo to 117 bpm.P E RVA S I V E C O M P U T I N G I N S P O R T S Pace-fixing mode We use time stretching to create a constant tempo by transforming a song if its tempo is different from the desired one.computer. denoted by tm. If the song can’t be stretched enough to accomwww. Pace-matching mode To match the tempo to the user’s stride frequency. The system selects other songs if it can no longer match the tempo to the user’s stride frequency. We assume that the music and the user’s stride are in step. Pace-influencing mode To motivate users to keep their heart rate within a certain zone. we define the song for playback as the one whose original tempo comes closest to the desired playout tempo. The system repeats these steps until the user has reached the exercise goal. 4. For instance. Therefore. Match the tempo to the user’s stride frequency to ensure the connection between them. For a song with a tempo of 100 bpm. When considering a 100-bpm song with a stretch range of 85–125 and an assumed Tm = 500 msecs. 2. In step 2. which specifies the time that a maximal change in tempo as defined by the song’s stretch range should take.6 The system then transforms the desired stride frequency into a desired tempo. we know that heart rate will increase or decrease proportionally with exercise intensity. we consider all integer multiples or integral divisions of stride frequency as possible tempo matches. The system matches the tempo to stride frequency by using pace-matching mode. We can’t perform time stretching with impunity for extreme scale factors because music that’s modified too extensively sounds awkward. the system compares the user’s heart rate to the heart rate goal for that exercise and determines the difference percentage. Wait for heart rate stabilization. The assumption is that. when the change takes a long time (more than 10 seconds). the stretch border closest to the stride frequency. The parameter that specifies this response time in pace-matching mode is Tm. the user will keep his or her movements in time with later changes in tempo in Step 3. After the time stretch. The system requires the new song to have a stretch range that includes the current tempo and stride frequency to allow for a transition without abrupt tempo changes. In principle. and the system will speed it up to 175 bpm. where max denotes the maximally allowed change in tempo as defined by a song’s stretch range. Typical maximal values for the scaling factors are -15 percent to +25 percent. From sports physiology. the system needs to play the music faster or slower using time stretching. is supplied with two points tm(0) = 0 and tm( max) = Tm. Obviously. by matching tempo to stride frequency in Step 1. If the user starts to run at a stride frequency of 175 spm. It predicts the required stride frequency by applying this percentage directly to the current stride frequency. a hiccup in the music playback is evident. the system uses a four-step method: 1. we use a standard time-domain technique based on synchronous overlap-and-add.org/pervasive . happen faster than real time for online stretching and playback. from that. 30 seconds) to reduce repetitive song changing over time. Time stretching involves shortening or lengthening an audio data file without pitch modifications according to a scale factor. which can be an integer multiple or integral division of the desired stride frequency.

500. We used the mean squared error to assess the match between stride frequency and tempo for each response time. it can predict how the heart rate (and hence the desired tempos) will progress in the next 30 seconds. We couldn’t arrive at statistically sound findings when slowing down running. Six healthy recreational runners (two females and four males from 22 to 26 years old) took part in the experiments. Response time for pace-matching mode To determine the optimal response time for pace-matching mode.000 msecs.500. it waits for the user’s heart rate to reach a steady state before making additional changes. abrupt speed changes made runners vary their stride frequency and stride length in erratic ways. We varied the speed from 8 km/hour to 12 km/hour and vice versa. the system adapted music to their stride frequency. The mean squared error favored short response times in the range of 0 to 500 msecs. It effectively minimizes the number of song changes needed. It appeared to us that participants largely based their judgments on the extent to which the music kept in time with their steps. Meanwhile. The user can display the performance data on the PC for inspection. the system keeps a history of heart rate measurements.6 Feedback stage: After exercising After exercising. in step 4. After each tempo change. Finally. the user can collect and store performance data. in which a single song with a steady original tempo of 120 bpm played through in-ear headphones. when the system has propagated the change. The runners tended to rate short and long response times higher than medium response times when speeding up running. where max denotes maximally allowed change in tempo as defined by the song’s stretch range. This function. User experiments As an informative evaluation. The system uses the stride-frequency data that it logged during the performance of the same exercise over several training sessions to determine an optimal set of tempos and songs for that particular exercise by means of Bayesian inference. and we manipulated the response times Tm from 0. we asked the participants to rate on a scale of 1 (very negative) to 5 (very positive) to what extent they appreciated the music change as a background to their running. 5. We instructed the participants to adjust their stride frequency to the treadmill’s speed. we conducted user experiments for finding the preferred response time parameter Tm for the pace-matching mode using a treadmill and the preferred propagation time parameter Ti for the pace-influencing mode. As in the pace-matching mode. 180 170 160 150 140 130 40 50 60 70 80 90 100 110 120 130 140 150 160 170 180 190 200 210 Time (secs) or she is under.modate the change in tempo. the system doesn’t carry out a change in tempo immediately. Pace matching mode: music playout tempo is adjusted to the runner’s stride frequency with different response times. Instead. From that. the heart rate will reach a plateau. 2. In addition. and advice on adapting the training method.or over-performing according to the training scheme. we had the runners run on a TechnoGym RunRace HC1200 treadmill. This prediction provides valuable information on what song is best to choose the next time a change is required for playback.000. and the list of songs played on a personal computer. As figure 3 shows. the system changes songs in the same manner as in pace-matching mode.000. When the user holds a submaximal exertion intensity constant. It’s up to the user to identify whether he JULY–SEPTEMBER 2005 Stride frequency (spm)/music playout tempo (bpm) Figure 3. These data can be used for visual inspection and system learning purposes. short response PERVASIVE computing 15 . In step 3. motivation. This synchronization was hard to achieve because using a treadmill produced undesirable effects. The user might not follow an abrupt change in tempo because of ignorance or fatigue. the training program used. is supplied with two points ti(0) = 0 and ti( max) = Ti. denoted by ti. the system uses a linear function to calculate the actual time to change from a given tempo to a new one. the system takes into account a propagation time. The system-learning aspects concern determining what music tempos will best accompany the performance for a given exercise. 1. and 10.

nl/~heiner.46. Contact him at Dept. We used the mean squared error to assess the match between the tempo and the stride frequency. computer. decentralized and context-based semantic systems. h t t p : / / i s m i r 2 0 0 3 . REFERENCES 1. His research interests include human-computer interaction and music informatics and technology. We must also investigate using music or audio reproduction properties other than tempo and their benefits in sport exercise motivation. Steffen Pauws is a senior scientist at Philips Research where he works on signal processing.com. Holstlaan 4. 6. He’s working on his M. Academic Press. E. 5656 AA Eindhoven. situations in which the music might adapt to the runner while the runner is adapting to the music need further scrutiny. we achieved the best synchronization performances with a propagation time of at least 20 seconds. The need to detect heart-rate stabilization is essen- 4. M. Costill. 1997. Human Kinetics. steffen. Rm. John Wiley and Sons.org/publications/dlib. It appeared that participants had difficulties synchronizing with the music if 16 PERVASIVE computing the tempo changes were done with propagation times of 5 and 10 seconds. n e t / p a p e r s / McKinney.D. 1. He is the author of Information Sharing on the Semantic Web and editor of a forthcoming book. DAFX: Digital Audio Effects. although achieving a heart rate plateau takes more time for more intense exercise levels.computer. Hoos and D. For instance. McKinney and J. by using stereo position parameters.PDF. Contact him at Philips Research Labs Eindhoven.Sc.org/pervasive . 5656 AA. Holstlaan 4 (WY21). we instructed the runners to run in time with the music by varying their step/stride frequency.l. 5. Heiner Stuckenschmidt is a senior researcher in the Knowledge Representation and Reasoning group at the Vrije Universiteit Amsterdam. tial to one of the playback modes.wijnalda@redant. eds. F uture work will focus on improving the complete system in a usercentered design methodology and validating its usability.vu. Jan van Herk and Tijn Schuurmans at Philips Research.H. www.. g.wijnalda. Acoustical Soc. and the use of Semantic Web technologies for the development of intelligent information systems for medical and scientific applications. combinatorial optimization. Physiology of Sport and Exercise. Eindhoven. E..com. We varied the tempo from 110 to 150 bpm and vice versa to reflect desired stride frequencies with the same values. Scheier. He received his PhD in speech processing and multimodal interfaces from the University of Genova. and 20 seconds.F. 103. As indicated by the mean squared error.pauws@philips. Fabio Vignoli is a senior scientist at Philips Research where he works on music related applications and user interaction topics.vu. The Netherlands. Breebaart.. the music can play in front of or behind the runner. “Tempo and Beat Analysis of Acoustical Musical Signals. In each condition. 4th Int’l Conf. The combined use of stride frequency and stride length when changing running speed or when minimizing fatigue need to be better understood for further algorithmic improvement. The Netherlands. He received his PDEng degree in Software Technology and his PhD in computing science and human-computer interaction from the Technische Universiteit Eindhoven. 1998.” Proc. For instance. of Mathematics and Computer Science Vrije Universiteit Amsterdam De Boelelaan 1081a 1081 HV Amsterdam. “Features for Audio and Music Classification. H. heiner@cs. 3rd ed. The Netherlands. of America. usefulness. 10. Local Search in Combinatorial Optimization. Eindhoven. 2004. and user interfaces for music applications. thesis on Interactive Music for Sports at Philips Research.nl/g. 2. respectively. John Wiley and Sons.06 Prof. 1993. Contact him at Philips Research Laboratories.” J. Bldg. Propagation time for pace-influencing mode To determine the optimal propagation times for pace-influencing mode.redant. Lenstra. J. www. 588–601. The Netherlands.nl. Foundations of Constraint Satisfaction. Aarts and J. 2002. we varied the propagation time Ti between 5. Bldg.l.L.06 Prof. Music Information Retrieval (ISMIR).K. 3. they showed a jerky stepping synchronization behavior in response to the tempo change. Johns Hopkins Univ. please visit our Digital Library at www. vol. Willmore and D. Tsang.H. U. 5656 AA. We also need to further address some issues. For more information on this or any other computing topic.cs.P E RVA S I V E C O M P U T I N G I N S P O R T S the AUTHORS Gertjan Wijnalda is a student in artificial intelligence with a specialization on intelligent audio and media at the Vrije Universiteit Amsterdam. Holstlaan 4. WY-2. Prof. times track changes in stride frequencies more accurately than longer response times do. 2003. WY-2. fabio. WY 2. www. ACKNOWLEDGMENTS Thanks to the members of the Sports Coach project—in particular. pp. His research interests include practical knowledge representation and non-standard reasoning for semantic search and information integration on the Web. Zölzer.vignoli@philips. Bainbridge. Semantic Web and Peer-to-Peer. Contact him at Philips Research Labs Eindhoven. which might motivate the runner to accelerate or decelerate. and desirability in a conclusive user evaluation. E.nl. 151–158. i s m i r. pp. no.

JULY–SEPTEMBER 2005 PERVASIVE computing 17 .