Skinning/Enveloping

Gears of War 2

Epic Games / ign.com

CS-C3100 Fall 2019 – Lehtinen !1
Skinning
• We know how to animate a
bone hierarchy
– Change the joint angles, i.e.,
bone transformations, over
time (keyframing)

Epic Games
CS-C3100 Fall 2019 – Lehtinen !2
Skinning
• We know how to animate a
bone hierarchy
– Change the joint angles, i.e.,
bone transformations, over
time (keyframing)
• Embed a skeleton into a

Epic Games
detailed character mesh
• Bind skin vertices to bones
– Animate skeleton, skin will
move with it
– But how?
CS-C3100 Fall 2019 – Lehtinen !3
Skinning
• Need to infer how skin deforms
from bone transformations.
• Most popular technique:

Skeletal Subspace Deformation
(SSD), or simply skinning
– Other aliases
• vertex blending
• matrix palette skinning
• linear blend skinning

CS-C3100 Fall 2019 – Lehtinen From wikipedia !4

SSD / Skinning
• Each bone “deforms

the space around it” (rotation, translation)

CS-C3100 Fall 2019 – Lehtinen !5

SSD / Skinning
• Each bone “deforms

the space around it” (rotation, translation)
– What if we attach each

vertex of the skin to a single bone?

CS-C3100 Fall 2019 – Lehtinen !6

SSD / Skinning
• Each bone “deforms

the space around it” (rotation, translation)
– What if we attach each

vertex of the skin to a single bone?
• Skin will be rigid, except at joints where it will
stretch badly

CS-C3100 Fall 2019 – Lehtinen !7

SSD / Skinning
• Each bone “deforms

the space around it” (rotation, translation)
– What if we attach each

vertex of the skin to a single bone?
• Skin will be rigid, except at joints where it will
stretch badly
– Let’s attach a vertex to many bones at once!
• In the middle of a limb,

the skin points follow the bone rotation
(near-)rigidly
• At a joint, skin is deformed according to a
“weighted combination” of the bones
CS-C3100 Fall 2019 – Lehtinen !8
Examples

Colored
triangles are
attached to 1
bone
Black
triangles are
attached to
more than 1

Note how
they are near
joints
James & Twigg 2005 CS-C3100 Fall 2019 – Lehtinen !9
Examples

Colored
triangles are
attached to 1
bone
Black
triangles are
attached to
more than 1

Note how
they are near
joints
James & Twigg 2005 CS-C3100 Fall 2019 – Lehtinen !10
Vertex Weights
• We’ll assign a weight wij

for each vertex pi for each bone Bj.
– “How much vertex i should move with bone j”

CS-C3100 Fall 2019 – Lehtinen !11

Vertex Weights
• We’ll assign a weight wij

for each vertex pi for each bone Bj.
– wij = “How much vertex i should move with bone j”
– wij = 1 means pi is rigidly attached to Bj.

CS-C3100 Fall 2019 – Lehtinen !12

 Vertex Weights
• We’ll assign a weight wij

for each vertex pi for each bone Bj.
– wij = “How much vertex i should move with bone j”
– wij = 1 means pi is rigidly attached to Bj.

Wang & Phillips 2002 CS-C3100 Fall 2019 – Lehtinen !13

From Assignment 3

CS-C3100 Fall 2019 – Lehtinen !14

Vertex Weights
• We’ll assign a weight wij

for each vertex pi for each bone Bj.
– wij = “How much vertex i should move with bone j”
– wij = 1 means pi is rigidly attached to Bj.
• Weight properties
– Usually want weights to be non-negative Why?

CS-C3100 Fall 2019 – Lehtinen !15

Vertex Weights
• We’ll assign a weight wij

for each vertex pi for each bone Bj.
– wij = “How much vertex i should move with bone j”
– wij = 1 means pi is rigidly attached to Bj.
• Weight properties
– Usually want weights to be non-negative
– Also, always want the sum over X all bones

to be 1 for each vertex, i.e., wij = 1 8i Why?
<latexit sha1_base64="cfYuZ1rBhY9vF+XCSVplrO2ykIs=">AAACCnicbZDLSsNAFIYnXmu9RV26GS2Cq5KIoBuh6MZlBXuBJoTJZNJOO5nEuSgldO3GV3HjQhG3PoE738Zpm4W2/jDw8Z9zOHP+MGNUKsf5thYWl5ZXVktr5fWNza1te2e3KVMtMGnglKWiHSJJGOWkoahipJ0JgpKQkVY4uBrXW/dESJryWzXMiJ+gLqcxxUgZK7APPKmToA8fgpz2R/ACutC70yiCXpwKxBikgV1xqs5EcB7cAiqgUD2wv7woxTohXGGGpOy4Tqb8HAlFMSOjsqclyRAeoC7pGOQoIdLPJ6eM4JFxImh2m8cVnLi/J3KUSDlMQtOZINWTs7Wx+V+to1V87ueUZ1oRjqeLYs2gSuE4FxhRQbBiQwMIC2r+CnEPCYSVSa9sQnBnT56H5knVNXxzWqldFnGUwD44BMfABWegBq5BHTQABo/gGbyCN+vJerHerY9p64JVzOyBP7I+fwCWlZmK</latexit>
j

CS-C3100 Fall 2019 – Lehtinen !16

Vertex Weights
• We’ll assign a weight wij

for each vertex pi for each bone Bj.
– wij = “How much vertex i should move with bone j”
– wij = 1 means pi is rigidly attached to Bj.
• Weight properties
– Usually want weights to be non-negative
– Also, always want the sum over X all bones

to be 1 for each vertex, i.e., wij = 1 8i
<latexit sha1_base64="cfYuZ1rBhY9vF+XCSVplrO2ykIs=">AAACCnicbZDLSsNAFIYnXmu9RV26GS2Cq5KIoBuh6MZlBXuBJoTJZNJOO5nEuSgldO3GV3HjQhG3PoE738Zpm4W2/jDw8Z9zOHP+MGNUKsf5thYWl5ZXVktr5fWNza1te2e3KVMtMGnglKWiHSJJGOWkoahipJ0JgpKQkVY4uBrXW/dESJryWzXMiJ+gLqcxxUgZK7APPKmToA8fgpz2R/ACutC70yiCXpwKxBikgV1xqs5EcB7cAiqgUD2wv7woxTohXGGGpOy4Tqb8HAlFMSOjsqclyRAeoC7pGOQoIdLPJ6eM4JFxImh2m8cVnLi/J3KUSDlMQtOZINWTs7Wx+V+to1V87ueUZ1oRjqeLYs2gSuE4FxhRQbBiQwMIC2r+CnEPCYSVSa9sQnBnT56H5knVNXxzWqldFnGUwD44BMfABWegBq5BHTQABo/gGbyCN+vJerHerY9p64JVzOyBP7I+fwCWlZmK</latexit>
j

• This means translation independence.

• (Again, a partition of unity – remember spline bases?)
CS-C3100 Fall 2019 – Lehtinen !17
Vertex Weights cont’d
• We’ll assign a weight wij

for each vertex pi for each bone Bj.
– wij = “How much vertex i should move with bone j”
– wij = 1 means pi is rigidly attached to Bj.
• We’ll limit the number of bones N that can
influence a single vertex
– N=8 bones/vertex is a usual choice
– Why?

CS-C3100 Fall 2019 – Lehtinen !18

Vertex Weights cont’d
• We’ll assign a weight wij

for each vertex pi for each bone Bj.
– wij = “How much vertex i should move with bone j”
– wij = 1 means pi is rigidly attached to Bj.
• We’ll limit the number of bones N that can
influence a single vertex
– N=8 bones/vertex is a usual choice
– Why? You most often don’t need very many.
– Also, storage space is an issue.
– In practice, well store N (bone index j, weight wij)
pairs per vertex.CS-C3100 Fall 2019 – Lehtinen !19
How to compute
vertex positions?

Epic Games
CS-C3100 Fall 2019 – Lehtinen !20
Linear Blend Skinning: Basic Idea
• Step 1: Transform each vertex pi with each bone
as if it was tied to it rigidly.

CS-C3100 Fall 2019 – Lehtinen !21

Linear Blend Skinning: Basic Idea
• Step 1: Transform each vertex pi with each bone
as if it was tied to it rigidly.
• Step 2: Then blend the results using the weights.

CS-C3100 Fall 2019 – Lehtinen !22

Computing Vertex Positions
• Step 1: Transform each vertex pi with each bone
as if it was tied to it rigidly.
• Step 2: Then blend the results using the weights.

p’ij is the vertex i

“ ”
transformed using
pij = T j pi bone j.
Tj is the current
transformation of
pi = wij pij bone j.
j p’i is the new
skinned position of
vertex i.
CS-C3100 Fall 2019 – Lehtinen !23
Computing Vertex Positions
• Vertex p0 has
Rest (“bind”) pose
weights
w01=0.5,
Bone 1: T1 Bone 2: T2
w02=0.5
p0
“Skin”

CS-C3100 Fall 2019 – Lehtinen !24

Computing Vertex Positions
• Vertex p0 has
Rest (“bind”) pose
weights
w01=0.5,
Bone 1: T1 Bone 2: T2
w02=0.5
p0 • Transform by
After rotations “Skin”
T’1 and T’2
Bo yields p’01, p’02
ne T’2
1: 2 :
T’ n e
1 Bo

p’01 p’02
CS-C3100 Fall 2019 – Lehtinen !25
Computing Vertex Positions
• Vertex p0 has
Rest (“bind”) pose
weights
w01=0.5,
Bone 1: T1 Bone 2: T2
w02=0.5
p0 • Transform by
After rotations “Skin”
T’1 and T’2
Bo yields p’01, p’02
ne T’2
:
1:
T’ n e 2 • the new position
1 Bo is p’0=

0.5*p’1 +
p’01 p’0 p’02 0.5*p’2
CS-C3100 Fall 2019 – Lehtinen !26
Computing Vertex Positions
• Vertex p0 has
Rest (“bind”) pose
weights
w01=0.5,
Bone 1: T1 Bone 2: T2
w02=0.5
p0 • Transform by
After rotations “Skin”
T’1 and T’2
Bo yields p’01, p’02
ne : T 2
1: e 2 • the new position
T’
Bon
1
is p’0=

“Skin” 0.5*p’1 +
p’01 p’0 p’02 0.5*p’2
CS-C3100 Fall 2019 – Lehtinen !27
SSD is Not Perfect
q0

p0
After rotations

CS-C3100 Fall 2019 – Lehtinen !28

 Questions?
• Next, explanation of

the mysterious quotes..

Remedy Entertainment, Microsoft Game Studios, IGN.com

CS-C3100 Fall 2019 – Lehtinen !29
Bind Pose
• We are given a skeleton and a

skin mesh in a default pose
– Called “bind pose”
– Undeformed vertices pi are given

in the object space of the skin

okino.com
CS-C3100 Fall 2019 – Lehtinen !30
Bind Pose
• We are given a skeleton and a

skin mesh in a default pose
– Called “bind pose”
– Undeformed vertices pi are given

in the object space of the skin
• Previously we conveniently

forgot that in order for

p’ij = Tj pi to make sense,

coordinate systems must

match up.
okino.com
CS-C3100 Fall 2019 – Lehtinen !31
Bind Pose cont’d
• In the rigging phase, we
 Bn
line the skeleton up with the

undeformed skin.
– This gives some “rest pose”
 ....
bone transformations Bj
B2

B1

okino.com
CS-C3100 Fall 2019 – Lehtinen !32
Bind Pose cont’d
• In the rigging phase, we
 Bn
line the skeleton up with the

undeformed skin.
– This gives some “rest pose”
 ....
bone transformations Bj
• We then figure out the vertex
 B2
weights wij.
– How? Usually paint by hand!
– Pinocchio was the algorithm
 B1
that does this automatically
okino.com
CS-C3100 Fall 2019 – Lehtinen !33
Bind Pose cont’d
• When we animate the model,

the bone transformations

Tj change.

okino.com
CS-C3100 Fall 2019 – Lehtinen !34
Bind Pose cont’d
• When we animate the model,

the bone transformations

Tj change.
– What is Tj? It maps from the

local coordinate system of

bone j to object space.
• Remember hierarchical modeling!

okino.com
CS-C3100 Fall 2019 – Lehtinen !35
Bind Pose cont’d
• When we animate the model,

the bone transformations

Tj change.
– What is Tj? It maps from the

local coordinate system of

bone j to object space.
• To be able to deform pi according

to Tj, we must first express pi in the local

coordinate system of bone j.
– This is where the bind pose

bone transformations Bj come in.
okino.com
CS-C3100 Fall 2019 – Lehtinen !36
Bind Pose cont’d
• To be able to deform pi

according to Tj, we must

first express pi in the

local coordinate system of bone j.
– This is where the bind pose

bone transformations Bj come in.
⇥
pij = T j B j pi 1

This maps pi from bind pose object space to the

local coordinate system of bone j using B-1j, and
then to deformed object space using Tj.
okino.com
CS-C3100 Fall 2019 – Lehtinen !37
Bind Pose cont’d

⇥
pij = T j B j pi 1

This maps pi from bind pose object space to the

local coordinate system of bone j using B-1j, and
then to deformed object space using Tj.

What is Tj B-1j? It is the relative

change between the bone
transformations between the
current and the bind pose.
CS-C3100 Fall 2019 – Lehtinen !38
 What is the
Bind Pose cont’d transformation
when the
model is still in
⇥
pij = T j B j pi 1 bind pose?

This maps pi from bind pose object space to the

local coordinate system of bone j using B-1j, and
then to deformed object space using Tj.

What is Tj B-1j? It is the relative

change between the bone
transformations between the
current and the bind pose.
CS-C3100 Fall 2019 – Lehtinen !39
 What is the
Bind Pose cont’d transformation
when the
model is still in
⇥
pij = T j B j pi 1 bind pose?
The
identity!
This maps pi from bind pose object space to the
local coordinate system of bone j using B-1j, and
then to deformed object space using Tj.

What is Tj B-1j? It is the relative

change between the bone
transformations between the
current and the bind pose.
CS-C3100 Fall 2019 – Lehtinen !40
Questions?
• Interlude: Pinocchio

CS-C3100 Fall 2019 – Lehtinen !41

Skinning Pseudocode
• Do the usual forward kinematics
– maybe quaternion interpolation for rotations
– get a matrix Tj(t) per bone
• For each skin vertex pi
⇥
pi = wij T j (t)B j pi 1

CS-C3100 Fall 2019 – Lehtinen !42

Skinning Pseudocode
• Do the usual forward kinematics
– maybe quaternion interpolation for rotations
– get a matrix Tj(t) per bone
• For each skin vertex pi
⇥
pi = wij T j (t)B j pi 1

Remember from Lecture 2:


Normals must be treated differently!

CS-C3100 Fall 2019 – Lehtinen !43

Skinning Pseudocode
• Do the usual forward kinematics
– maybe quaternion interpolation for rotations
– get a matrix Tj(t) per bone
• For each skin vertex pi
⇥
pi = wij T j (t)B j pi 1

j
• Inverse transpose for normals!
⇤ ⇥ T
⇥
ni = wij T j (t)B j 1
ni
j
CS-C3100 Fall 2019 – Lehtinen !44
Skinning Pseudocode
• Do the usual forward kinematics
– maybe quaternion interpolation for rotations
– get a matrix Tj(t) per bone
• For each skin vertex pi
⇥
pi = wij T j (t)B j pi 1

j
• Note that the weights are constant over time
– Only a small number of matrices change
– This enables implementation on GPU “vertex shaders”

(little information to update for each frame)
CS-C3100 Fall 2019 – Lehtinen !45
Questions?

CS-C3100 Fall 2019 – Lehtinen !46

Hmmh...
• This is what we do to get deformed positions
⇥
pi = wij T j (t)B j pi 1

CS-C3100 Fall 2019 – Lehtinen !47

Hmmh...
• This is what we do to get deformed positions
⇥
pi = wij T j (t)B j pi 1

• But wait... ⇥
⇧
pi =
⇥ ⇤ wij T j (t)B j 1⌅
pi
j

CS-C3100 Fall 2019 – Lehtinen !48

Hmmh...
• This is what we do to get deformed positions
⇥
pi = wij T j (t)B j pi1

• But wait... ⇥
⇧
pi =
⇥ ⇤ wij T j (t)B j 1⌅
pi
j

• This is exactly what I warned you of earlier:

blending matrices entry-by-entry
CS-C3100 Fall 2019 – Lehtinen (!!!) !49
Indeed... Limitations
• Rotations really need to be combined differently
(quaternions!)

• From: Pose Space Deformation: A Unified Approach to Shape Interpolation and

• Skeleton-Driven Deformation
• J. P. Lewis, Matt Cordner, Nickson Fong
CS-C3100 Fall 2019 – Lehtinen !50
Dual Quaternion Skinning

✘
✔ CS-C3100 Fall 2019 – Lehtinen !51
Usual Solution
• Have the artists deal with it O:-)

• In practice, build a rig that has extra bones near

joints that move in sync to counter the artifacts.
– Tedious, but this is what people often do.
– Need sophisticated animation controls to drive this.

• Cool paper that does this automatically:


Kavan, Collins, O’Sullivan: Automatic
Linearization of Nonlinear Skinning, I3D 2009
CS-C3100 Fall 2019 – Lehtinen !52
Real-time enveloping with rotational regression
Wang, Pulli, Popovic
We learn a fast model from exported examples.

Black Box Exported Examples Fast Model

Simulation
(skeleton-mesh pairs) Our method

Slide from Rob Wang

Figuring out the Weights
• Usual approach: Paint them on the skin.
• Can also find them by optimization from example
poses and deformed skins.
– Wang & Phillips, SCA 2002

CS-C3100 Fall 2019 – Lehtinen !54

Super Cool: Automatic Rigging
• When you just have some reference skeleton
animation (perhaps from motion capture) and a
skin mesh, figure out the bone transformations
and vertex weights!
• Ilya Baran, Jovan Popovic: Automatic Rigging
and Animation of 3D Characters,

SIGGRAPH 2007
– http://www.mit.edu/

~ibaran/autorig/
• You saw this earlier
CS-C3100 Fall 2019 – Lehtinen !55
The Other Direction
• When you have no skeleton, but a source
animation for the full mesh (not so common)

CS-C3100 Fall 2019 – Lehtinen !56

That’s All for Today!

Remedy / Microsoft / ign.com

• Further reading
– http://www.okino.com/
conv/skinning.htm

• Take a look at any

video game – basically
all the characters are
animated using SSD/
skinning.

CS-C3100 Fall 2019 – Lehtinen !57