TM

LiveMove 2
Director’s Cut

TM
AiLive
www.AiLive.net
Machine Learning for Games
 On the path to creativity. . .

c 2008 AiLive Inc.

All rights reserved. No part of this manual may be reproduced or transmitted in any form or by any means
without the written consent of AiLive Inc.

AiLive, LiveMove, LiveMove Pro, LiveMove 2 and LiveCombat are either registered trademarks or
trademarks of AiLive Inc. in the United States and/or other countries. Other product and company names
mentioned herein may be the trademarks of their respective owners.
 5

Contents
1 What is LiveMove 2? 6
1.1 Core capabilities: Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.2 Major new features: Tracking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.3 Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2 Game Production with LiveMove 2 8

2.1 Working with Motion Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.2 Working with Tracking Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

3 LiveMove Licensing 10

4 Conclusion 11
6 1 WHAT IS LIVEMOVE 2?

1 What is LiveMove 2?
AiLive is proud to announce LiveMove 2: Motion Recognition and Tracking for Games, our most advanced
motion processing product yet. LiveMove 2 provides full recognition and tracking support for Nintendo’s new
TM
MotionPlus accessory, and continues to provide superb motion recognition for the original Wii Remote.
Motion tracking together with motion recognition offers many exciting new possibilities for revolutionary
motion control. Recognition tells you what the player is doing, while tracking tells you how the player is
doing it. With LiveMove 2, the developer has both the semantics and the syntax of every move while it is in
progress. Such complete information about motions makes it practical, not just possible, for developers to
create an entirely new level of natural, responsive and immersive motion control experiences for games.
Please see www.ailive.net/lm2.html for information on the free evaluation version, movies, and other infor-
mation on LiveMove 2.

1.1 Core capabilities: Recognition

LiveMove products excel at motion recognition. Motion recognition can tell you what move a player did,
but has only the vaguest idea of how that move was executed. LiveMove makes it so easy and fast to build
recognizers that anyone can do it. Literally.
All LiveMove products do the following:

• LiveMove-enabled games can recognize nearly any motion performed with a 3D motion sensing device.

• You define motions by giving examples – no coding required.

• LiveMove automatically recognizes the motions you define.

• The recognition is accurate across different players.

LiveMove Pro is the Gold Standard for motion recognition with the Wii Remote. On the same 3D device,
LiveMove 2 has identical motion recognition capabilities as LiveMove Pro, but with significantly improved
memory performance. AiLive is currently offering both products to the developer community.
Accurate recognition, or “classification” is available within just a few frames. Recognition is so responsive
that the game will often know what you are doing as soon as you start moving!
LiveMove products do not need to be told when a player’s motion starts and ends. Instead, LiveMove can tell
that to the game. So with a LiveMove-enabled game, the player can simply pick up the Wii Remote and start
moving without the need to press buttons.
There are many other motion recognition features for LiveMove Pro. If motion recognition on a 3D device is
all you need for your games for the forseeable future, please see http://www.ailive.net/liveMovePro.html for
more information about the product and a free evaluation version.

1.2 Major new features: Tracking

With the MotionPlus, there is enough motion sensing data available to go to the next step: motion tracking.
Motion tracking tells you exactly how the player moved the remote, without actually knowing what move the
player did (i.e. was that a forehand or a backhand?). Motion tracking gives you a relative position (x, y, z),
and orientation (α, β, γ ) of your remote every frame. Full tracking data alone won’t stay accurate for long
due to drift (see Section 2), but it is simply amazing for short motions like sword slashes, punching and
throwing, grabbing, and so on.
LiveMove 2 combines motion tracking and motion recognition to give you some very powerful new capabili-
ties:
1.2 Major new features: Tracking 7

• Full support for MotionPlus and the classic Wii Remote.

LiveMove 2 provides complete motion recognition and tracking solutions for the “MotionPlus controller”
(a Wii Remote with the MotionPlus accessory). Having collaborated on the MotionPlus hardware and
SDK design, AiLive is in an unique position to offer exceptional software support for it through
LiveMove 2. LiveMove 2 also fully supports motion recognition on the classic Wii Remote.

• 6 DOF position and orientation tracking.

With LiveMove 2 you can track both 3D positions and 3D orientations simutaneously, for short periods
of time. This means your on-screen feedback to the player can be immediate, and as close to what is
actually happening with the controller as you want.

The first clip of the LiveMove 2 movie shows some simple tracking. The controller in the
‘game’ is moving in exactly the same way as it is in the presenter’s hand. The following
swordplay clips give you an even better feeling for how responsive and fun motion
tracking at this level can be. For the first time players can experience “WYDIWYS”
(what you do is what you see), or one-to-one (1:1) feedback. When you get your first
chance to work with LiveMove 2 and the MotionPlus, you’ll be amazed by how good this
feels for short motions.

• Improved recognition using MotionPlus data.

LiveMove 2 uses the additional MotionPlus data to achieve even better recognition rates on player mo-
tions. Typical recognition rates for MotionPlus data show a relative performance gain of 10-20% (and
up to 50% improvements) over their 3D counterparts. The recognizers are also significantly more ro-
bust across different users. This can make all the difference in some cases between usable and unusable
motion recognition. For example, consistently detecting sword strokes with differing angles done by
all types of players is nearly impossible with 3D data. The raw data is not sufficient. LiveMove 2 with
the MotionPlus make it easy to design robust swordplay.
• Recognition of two-handed, coordinated motions.
LiveMove 2 adds support for recognizing two-handed, coordinated motions. Coordinated motions can
be performed with two separate Wii Remote or MotionPlus controllers or one controller connected with
the Nunchuk. This capability is essential for recognizing skill moves that require two hands working
together, for example holding a big bowl of water, threading a needle, doing a tai-chi form, or playing
Simon-says.
• Re-designed lmMaker with new debugging facilities.
lmMaker is a Graphical User Interface (GUI) that is used for collecting examples and building rec-
ognizers. The lmMaker GUI now provides motion collection, classifier building and testing support
for a broad range of device types. The GUI is re-designed to be streamlined and more intuitive to use.
Additionally, it offers new debugging facilities:1

– Cross validation: Tools that provide realistic assessment of the recognition performance across
different players. Such tests are crucial for determining when to stop adding new training motions
to the recognizer and when to start fine-tuning it.
– Test motions: GUI support for collecting separate test motion sets on which the classifier can be
further tested and debugged.
– Pin-point debugging: Click on a test move or motion to get hints about why it is unrecognized
or mis-classified and what are the possible ways to fixing it.
– Views: Multiple views of the training or test motions for better understanding of the data.

LiveMove 2 gives you unprecedented control over recognition and tracking in your game. Use the complete
design flexibility for motion control to differentiate your games. Don’t just give your players whacks and
wiggles. The design space for free form motions in games is wide open, unexplored, and coveted by players.
1 This functionality will be part of the Q4’08 patch release.
8 2 GAME PRODUCTION WITH LIVEMOVE 2

1.3 Components
LiveMove 2 has four major components:

• libLM, a run-time library optimized for the Wii for motion recognition and tracking in real-time.
The libLM code interface is small and conceptually simple. To integrate libLM into your game
requires only a small amount of coding, at most 1 programmer for a day or so.

• lmMaker, a development-time application for the PC that builds classifiers from motion examples.
The lmMaker tool provides a GUI that lets anyone collect examples and build classifiers. No coding
skills are needed. For example, game designers can directly author moves.

• An extensive User Manual with introductory tutorial, some sample code, and some additional applica-
tions.

• AiLive support!

2 Game Production with LiveMove 2

2.1 Working with Motion Recognition

LiveMove solves the hard technical problems you face when interpreting motion sensor data. LiveMove
products take a machine learning approach to motion recognition. Motion recognizers are learned, not coded,
which has many advantages. The biggest is that anyone can create recognizers and test them in-game in
minutes. There is no wait time between wanting to test a new motion control scheme, and actually being able
to do so in your game.
Some other advantages of working with LiveMove 2 motion recognizers:

• Collect for both Wii Remote and MotionPlus simultaneously.

With LiveMove 2, you can collect all of your data once with MotionPlus accessories, then build recog-
nizers for the Wii Remote and for the MotionPlus without needing to collect more data. Of course, both
recognizers will still need testing.

• Fast, scalable and efficient

With up to 5% of your CPU on a given frame, LiveMove Pro can drive recognition for dozens of dif-
ferent motions in your game with 4 players simultaneously classifying on 4 remotes with 4 Nunchuk
controllers at 60 frames per second. With one player and one controller, there is practically no limit.
Total memory use for a good recognizer is typically several hundred kilobytes. If you need fewer
moves, recognizers will shrink accordingly.
LiveMove 2’s CPU requirements are similar, the dynamic memory performance is dramatically im-
proved, and total memory requirements are reduced. CPU performance is linearly related to the num-
ber of data streams needed for the recognizer. For example, 6D MotionPlus recognition is twice as
expensive as Wii Remote recognition, and two-handed coordinated MotionPlus recognition is another
doubling.

• Motion control is a reusable asset

LiveMove 2 is data-driven and modular. Adding a new move to a game is done by adding motion
examples for that move and rebuilding a classifier. You can refresh titles by downloading new motion
recognizers. Moves, motions and recognizers can be shared across different games.

• Lower risk, lower development costs, better results

2.2 Working with Tracking Data 9

Writing code to robustly recognize a set of moves in-house could easily employ one or more developers
full-time over the duration of a project. Without LiveMove, evaluating new ideas can take weeks,
assuming you can get over the fear of changing any of that magic code. One of the many developers
who’ve tried LiveMove tells it best:

”The code we currently have is messy, full of magic numbers, ridiculously long, and VERY
hard to understand and modify. Changing even the slightest thing could break everything in
unexpected ways.”

The design cycle for exploring motion control is so much simpler and quicker with LiveMove, it is
revolutionary. Want to make an action game with dozens of special moves, or a sports game with tons
of victory celebrations? You can. Your designs can be as ambitious as you want. Unsure which moves
feel right together? Once you’ve collected examples, you can try out different subsets of moves with
LiveMove within minutes.
LiveMove tools even let you test your ideas before any coding for a game begins.

• Finishing
LiveMove products are not just fast prototyping tools. They are also complete motion design packages
that can take you all the way through to shipping a game. There are dozens of LiveMove-enabled games
that have shipped or are in the final stages of doing so. Our customers range from the largest publishers,
to the smallest of indie developers. Once you are ready to finalize your classifier for production, you
will find LiveMove makes it as easy as possible.

2.2 Working with Tracking Data

Tracking data has different properties than motion recognition data, and needs to be handled differently.
LiveMove 2 gives you frame-level tracking. There is almost no lag time between the player moving the
controller, and your access to the corresponding position and orientation data. You’ll have it within 0 or 1
frames. Such immediacy is an extremely powerful tool for creating natural and immersive experiences. It
does come with a cost. One of the key limitations for any self-contained motion sensing device is accumulated
drift. No software can completely overcome this problem.
Devices like the Wii Remote and the MotionPlus are self-contained inertial sensing devices. They do not
depend on external signals or beacons to work. You will get valid lag-free motion sensing data without any
input from an external camera, GPS, or a DPD. But there are errors in every reading, and tracking errors
accumulate and grow rapidly. The good news is LiveMove 2’s patent pending machine learning and statistical
methods expose the full power of such devices to you. You can focus solely on the rocket science of creative
game development.
Raw, full tracking data for the MotionPlus and LiveMove 2 is good for 1 to 2 seconds between resets. This
is more than enough for astonishing new in-game experiences revolving around short actions like striking,
swinging, hitting, throwing, stabbing, slashing, jabbing, catching, pointing, and so on. LiveMove 2 can make
use of several types of resets (many of which can be nearly invisible in-game), and can improve raw tracking
performance by 10x or more.
The following will help you understand the power and limits of LiveMove 2 tracking, and how best to take
full advantage of it for your games.

• Drift Sources
Gravity, centripetal force and the Corialis effect all affect the accelerations on the sensors that we
depend on to measure player motion. We need to remove the effect of these forces from the sensor data
before we can track and report the player’s movement.
The sensors themselves and their control circuitry limit the maximum magnitudes and the precision
with which all such forces can be reported. Measurements also vary slightly from 1 device to the next.
10 3 LIVEMOVE LICENSING

If there is even a 1% error in our estimate of the direction of gravity, it leads to a 9.8cm/s 2 error in
acceleration, which can translate to ∼ 40cm of error in position after just a few seconds. These errors
accumulate over time, and are the key reasons why your controller will appear to drift the longer it
moves without a reset.
• Drift Control
Drift errors can never be removed in software. However, there are many ways to control or compensate
for drift. LiveMove 2 employs all of these techniques:

– External beacons: Take advantage of external sources of data whenever you have them. LiveMove 2
uses DPD data when it is available to stabilize the currently reported tracking data.
– Gravity: LiveMove 2 looks for cases where it can get a stable gravity reading for a fraction of
a second in order to reset and remove ongoing drift. Motion design that includes natural pauses
plays very strongly into this capability. For example, tennis, ping pong, swordplay and many
other activities involve being in a ready stance, executing a short motion, and going back to a
ready stance.
– Short motions: Drift grows over time. The shorter your motions, the better your tracking.
– User and Application knowledge: LiveMove 2 uses knowledge about both how humans typically
make motions and general constraints on human physical capabilities in order to help extract the
real motion of the controller from the different sources of accumulated drift.

• Relative 6 DOF tracking

The position and orientation that we report is relative to the player’s starting position. So for example,
if a player draws a “3” in the upstairs bedroom when facing the TV in the northern corner of her house,
and the same “3” downstairs facing the TV in the southern corner, the LiveMove 2 tracking data your
game receives will be identical.
• Personalized animation
You can use LiveMove 2’s tracking API to produce completely free-form procedural animations in game
that reflect everything the player is doing. This is the highest form of personalization.
Alternatively, you can start with beautiful, canned animations. You can then blend and control the
timing of them to allow progress through the animation to be determined by where the player’s arm is
based on tracking output. For example, a player could slow down midway through a sword stroke, or
even back off if they want to take it easy on their mom. Mom’s in-game sword will waver just as her
hand is in real life, showing the desperation she felt at the close call.
• Complementary power of motion recognition
With LiveMove 2, you can use tracking and recognition at the same time. Recognition can tell you
“topspin lob!”, while the tracking data is used to dress up the animation of the actual contact with the
tennis ball.

We can’t wait to see how you will use LiveMove 2 in your games!

3 LiveMove Licensing
AiLive is currently offering both products, LiveMove Pro and LiveMove 2, to the developement community.
So which one is right for you?
LiveMove 2’s licensing structure is similar to that of LiveMove Pro , but with a few additional options. See
AiLive’s website, or mail support@ailive.net for details.
LiveMove 2 and LiveMove Pro both give the licensed party the right to create LiveMove-enabled games for
five years. Which product to use comes down to whether or not you’ll want to take advantage of MotionPlus-
specific features at any point during those five years. Since LiveMove 2 does everything LiveMove Pro does,
 11

along with tracking and full MotionPlus support, if you may develop games for MotionPlus in your future,
you should choose LiveMove 2.

4 Conclusion
AiLive’s LiveMove brand of software products is the Gold Standard for developing motion controls on the
Wii . We make it possible to take advantage of all of the novelty of these devices, without risking the signifi-
cant technical and time-related hurdles.
It doesn’t matter if you’re a large studio or a small one. The limiting factor for motion control on the Wii is
now driven by the creativity of individuals. Anyone can begin exploring new motion controls in minutes.
Literally.
With LiveMove products, you’ll create motion recognizers at a pace that can run circles around the modern
game development cycle. With LiveMove 2, you can create motion controls for games that will knock the
socks off today’s players, while also knocking 6 months off the front end of your development cycle.