## Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

Bernd Bickel

1

Peter Kaufmann

1

M´ elina Skouras

1,2

Bernhard Thomaszewski

1,2

Derek Bradley

1

Thabo Beeler

1

Phil Jackson

3

Steve Marschner

4

Wojciech Matusik

1

Markus Gross

1,2

1

Disney Research Zurich

2

ETH Zurich

3

Walt Disney Imagineering

4

Cornell University

Figure 1: Left: photograph and scanned 3D geometry of a human face. We use a physics-based optimization process to design the geometry

of a synthetic skin in order to best match given target expressions. Right: ﬁnal animatronic ﬁgure with fabricated skin.

Abstract

We propose a complete process for designing, simulating, and fab-

ricating synthetic skin for an animatronics character that mimics the

face of a given subject and its expressions. The process starts with

measuring the elastic properties of a material used to manufacture

synthetic soft tissue. Given these measurements we use physics-

based simulation to predict the behavior of a face when it is driven

by the underlying robotic actuation. Next, we capture 3D facial ex-

pressions for a given target subject. As the key component of our

process, we present a novel optimization scheme that determines

the shape of the synthetic skin as well as the actuation parameters

that provide the best match to the target expressions. We demon-

strate this computational skin design by physically cloning a real

human face onto an animatronics ﬁgure.

Keywords: animatronics, physics-based simulation, computa-

tional material design, facial animation, optimization.

Links: DL PDF WEB VIDEO

1 Introduction

We are naturally intrigued by the prospect of creating virtual hu-

mans in the likeness of ourselves — and it is not far-fetched to

say that this is also a driving force for computer graphics research.

While the latter strives to photorealistically recreate human charac-

ters on a computer screen, animatronics aims at creating physical

robot characters that move and look like real humans. Both ﬁelds

share many similarities; however, animatronics faces harder chal-

lenges since it has to deal with the physical constraints of real ma-

terials and actuators in addition to the virtual modeling component.

The human body consists of articulated rigid structures (bones) and

soft tissue (e.g., ﬂesh and skin). Therefore, it seems natural for an-

imatronic characters to have a rigid articulated base and synthetic

soft tissue. Many impressive characters have been created in this

spirit, e.g., those in Disney World’s Hall of Presidents or “Gemi-

noids” [?], androids that closely resemble human beings. However,

creating such ﬁgures is still a difﬁcult and labor-intensive process

requiring manual work of skilled animators, material designers, and

mechanical engineers. Owing to its expressive power, the human

face is probably the most challenging part in this context. An ani-

matronic character has to produce a vast range of facial expressions,

each having different deformations and wrinkles. Manually design-

ing the shape and material properties of a single skin that is able to

achieve all these targets is clearly a formidable task.

The goal of this work is to automate this process, to increase the

realism of the resulting character and, ultimately, to create an an-

imatronic face that closely resembles a given human subject. In

order to accomplish this task, we capitalize on recent developments

from three areas in computer graphics: facial performance capture,

physics-based simulation, and fabrication-oriented material design.

Building on these foundations, we present a method of physical face

cloning — a novel process for computational modeling, optimiza-

tion and fabrication of synthetic skin for animatronic characters.

More speciﬁcally, our process comprises the following steps. First,

we capture elastic material properties for a range of possible syn-

thetic skin materials using a custom measurement system. We sub-

sequently capture a collection of different expressions for a given

target human face. As the central part of our pipeline, we then op-

timize the geometry of the skin and the actuation parameters of the

underlying animatronics device to provide the best match to the tar-

get human face. Although our method applies quite generally to a

broad range of possible animatronic devices, we validate the whole

process in this paper by fabricating synthetic silicone skins for a

speciﬁc articulated robot head. These skins are then animated, and

the resulting shapes are compared both to the optimized model and

to the real face.

The contributions of the research presented in this paper can be

summarized as follows:

• A method to optimize for synthetic skin geometry and actua-

tion parameters that provide the best match to a collection of

given target expressions,

• A complete process for automating the physical reproduction

of a human face on an animatronics device, including data

acquisition, physical simulation, optimization, fabrication and

validation,

• A comprehensive analysis and experimental validation of the

process and its individual steps.

2 Related Work

Physical face cloning is a complex task that intersects with numer-

ous research areas from computer graphics, applied mathematics,

and material sciences. Although we are not aware of existing work

to address this problem in its entirety, the individual components of

our system have diverse connections to previous research.

Face Capture. Owing to the expressive power of the human face

and our ability to infer a burst of information from the slightest mo-

tion, capturing, synthesizing and transferring facial expressions are

important research topics in computer graphics. There exists a mul-

titude of methods for capturing facial expressions, including tradi-

tional marker-based motion capture [?; ?], structured light systems

[?; ?], and passive markerless approaches [?; ?; ?]. Our acquisition

system uses the recent method of Beeler et al. [?] to capture short

performance sequences with high-resolution geometry and dense

temporal correspondences.

Simulation. Introduced to the graphics community by Terzopou-

los et al [?], it has become common to model elastically deformable

materials using continuum mechanics and ﬁnite elements. There

exists a plethora of dedicated bio-mechanical models for different

parts of the human body such as the face [?; ?; ?], the hand [?; ?],

the torso [?], various organs [?], or even the entire upper body [?].

Similarly, a large body of literature is concerned with modeling of

biological soft tissue (see, e.g., the work by Teran et al. [?] and

references therein). We do not strive to reproduce the mechanical

properties of biological tissue, but rather take an output-oriented

approach: we fabricate synthetic skin using silicone rubbers, which

are adequately modeled as elastic materials with isotropic and ho-

mogeneous properties. More speciﬁcally, we rely on a compress-

ible neo-Hookean material model and, on the computational side,

employ a ﬁnite element approach centered around linear tetrahedral

elements.

Measuring and Fitting Material Parameters. Closely related to

the choice of a constitutive law is the problem of ﬁnding a set of

parameters that best approximates a given reference model. This

reference data can originate either from an accurate computational

model [?; ?; ?] or, as in our case, from measurements of real-

world objects. In the context of computer graphics, several meth-

ods have been described for ﬁtting measured data to computational

models, including linear-elastic materials [?], nonlinear viscoelas-

tic soft tissue [?], or nonlinear heterogeneous materials [?]. Pai

et al. [?] describe a comprehensive system for capturing interac-

tions with real-world objects, including deformation response, sur-

face roughness and contact sounds. Our approach for determining

parameters of different silicone samples goes along similar lines:

we use a vision-based acquisition system to capture dense surface

displacement ﬁelds and determine optimal material parameters by

minimizing the displacement error of a corresponding ﬁnite ele-

ment approximation.

Fabrication-oriented Design. As a natural step to follow acqui-

sition and data-driven simulation, fabrication-oriented design is a

recent trend that is seeing increasing attention in computer graph-

ics. Several methods have been presented in this context, targeting

the reproduction of geometric features of papercraft [?] or plush

toys [?], general appearance properties [?; ?; ?], and also the elastic

properties of deformable media [?]. Our approach for fabrication-

oriented material design bears some similarities to this last refer-

ence, but there is also a crucial difference: whereas Bickel et al.

determine spatially varying material properties for ﬁxed geometry,

we fabricate synthetic skins with uniform material properties and

optimize their shape.

Shape Optimization. Engineering applications often require the

shape of a structure to be optimal with respect to given conditions.

Examples of such shape optimization problems include the com-

putation of aerodynamic proﬁles, the design of reﬂectors for lumi-

naires or the optimization of material interfaces. Such problems are

typically cast as constrained optimization problems and solved nu-

merically with ﬁnite elements (see, e.g., the textbook by Bucur and

Buttazzo [?] for an introduction). We optimize the shape of the

synthetic skin by minimizing an elastic energy with respect to rest

state positions, which is similar in spirit to methods for variational

shape adaptation [?]. But since it is generally difﬁcult to control

the adaptation of the undeformed conﬁguration, we use a dedicated

parametrization approach combined with a moving least squares in-

terpolation technique that allows us to modify the shape only in its

thickness direction.

Robotics & Animatronics. The majority of robots consist of

only rigid mechanical structures. However, robots that are cov-

ered with soft-tissue materials such as rubbers or silicones have

been built as well. Hara et al. [?] was one of the ﬁrst group

to build an anthropomorphic head to study human-robot interac-

tion. The face skin was made of silicone and it was driven by

SMA (shape memory alloy) actuators. The head was able to re-

produce six different expressions such as surprised, scared, an-

gry, disgusted, sad and happy. Android research, which involves

studying human-computer interaction, has been carried out by In-

telligent Research Laboratory at Osaka University. The group has

built a number of robotic characters with increasing capabilities.

Their recent humanoid robot Geminoid HI-1 [?] as well as it’s

successor Geminoid-DK have the realistic appearance of an adult

man. These robots also have silicone-based skin driven by actua-

tors placed underneath. KAIST and Hanson Robotics have devel-

oped a robot called Albert HUBO [?]. The robot’s head resembles

Albert Einstein and its underlying electro-mechanical actuation has

35 degrees of freedom. The skin is made of a sponge-like elastomer

which is very soft and requires very little force to deform. In con-

trast to our approach, the design process for all these robotic skins

is mainly manual. Shape variations are modeled by an artist, and

the implications of changes to the shape have to be tested in a time

consuming trial and error manner.

3 Overview

Our approach to physical face cloning comprises modeling, simula-

tion, design, and fabrication of synthetic soft tissue. The individual

steps of our processing pipeline are illustrated in Fig. 2.

We start by capturing facial expressions of a human subject using

an optical performance capture system that offers high-resolution

3D reconstructions including pores and wrinkles, as well as robust

temporal correspondence (Sec. 4). This data provides information

about the deformation behavior of the subject’s skin. In addition,

Face Meshes

Animatronic

Head

Skin Attachement

Paths

f(p

i

)=

Material

Sample

Material

Parameters

Acquisition Optimization

Actuation

Parameters

Material

Thickness

Fabrication

Final Skin

3D Printed

Mold

Figure 2: Physical face cloning pipeline.

we also determine the operational range of the animatronic head,

which consists of a set of skin attachment links actuated by motors.

We attach to each moving link a marker, which allows us to sample

a mapping of actuation parameters pact to 3D location and orien-

tation of each link in space. Interpolating these samples yields a

continuous function m(pact) that describes the path of the skin at-

tachment links (Sec. 7). Given the acquired example shapes of the

human face and the description of the animatronic head, the goal is

now to design synthetic soft tissue that matches the example shapes

as closely as possible.

Our base material for the synthetic skin is silicone, which of-

fers a wide range of stiffness that can be controlled by adjust-

ing the concentration of plasticizer in the compound. In order to

determine material parameters for the skin simulator, we numeri-

cally ﬁt our computational model to experimentally acquired force-

displacement samples of materials with different stiffnesses. We

model and simulate the deformation behavior using a non-linear ﬁ-

nite element method in combination with a neo-Hookean material

(Sec. 5). The forward simulation allows accurate prediction of the

deformed shape of a given synthetic skin and the resulting forces

induced by the moving skin attachment links.

In order to design a synthetic skin that best matches a desired tar-

get, we introduce a novel optimization process. While the outer

surface of the skin in the undeformed conﬁguration is given by a

3D scan of the neutral target pose, we propose to vary the inner

surface (thickness) of the skin to achieve spatially varying stiffness

and deformation behavior. Furthermore, we also optimize the ac-

tuation parameters of the animatronic device to ﬁnd the parameters

that best resemble the individual target expressions. All these de-

grees of freedom are handled in one uniform optimization frame-

work (Sec. 6).

We validate our simulation and optimization approach by fabricat-

ing various silicone objects using injection molding (Sec. 8). The

molds are created using a rapid prototyping 3D printer. We then

compare predicted simulation results and real-world behavior for

different deformation scenarios (Sec. 9).

4 Measurements and Acquisition

We rely on real-world data for modeling facial geometry and ex-

pressions as well as material behavior. The following paragraphs

brieﬂy describe our techniques for data acquisition and integration.

Face Scanning. When cloning a human face, matching the shape

of the face and how the skin deforms under different expressions is

one of the main challenges. We must reconstruct various 3Dexpres-

sions of the subject to be cloned, in order to seed the deformable

skin optimization. Our approach is to capture a short performance

of the target subject, leveraging the high resolution geometry acqui-

sition method of Beeler et al. [?]. The result is a detailed, compati-

ble mesh sequence with explicit temporal correspondence, allowing

us to analyze the deformation of a face under various expressions,

down to the level of individual wrinkles. In our case, we chose 8

expressive poses from the sequence for optimizing the skin param-

eters.

Measuring & Fitting Material Parameters. Silicone rubber is an

ideal base material for fabricating synthetic skin since it is easy to

process and offers a vast range of stretch resistance. The anima-

tronics device can only exert a limited force through its actuators,

which imposes an upper bound on the net stiffness of the synthetic

skin. Hence, we have to adjust the silicone composition such that

the net stiffness of the synthetic skin complies to these constraints.

To accomplish this task, we ﬁrst measure the force-deformation be-

havior of a collection of silicone samples with different amounts of

plasticizer. We then perform numerical optimization in order to de-

termine the parameters of our computational model that best match

the experimental data.

The measurement proceeds by pulling on small samples with con-

trolled forces and capturing the resulting deformation over time us-

ing stereo reconstruction [?]. With a set of applied loads and re-

sulting surface displacements determined, we optimize the material

parameters of our ﬁnite element solver in order to best approxi-

mate the measured stress-strain behavior. To this end, we start by

deﬁning an objective function O(p) =

i

| (˜ xi −xi(p)) |

2

that

measures the difference between simulated surface positions xi(p)

and their closest corresponding point of the captured surface ˜ xi.

The vector p holds the physical parameters of the material model,

which can be identiﬁed with Young’s modulus and Poisson’s ra-

tio (see Sec. 5). Given an initial guess p0 we iteratively compute

updated parameters in a simulated annealing process. In each itera-

tion, we compute the ﬁnite element solution x(pi) with the current

parameters, evaluate the objective function O(pi) and determine

updated parameters pi+1. This process is repeated until the objec-

tive function signals convergence.

As a validation for this numerical-experimental material ﬁtting pro-

cess, we compare the stress-strain curves obtained from a standard

testing procedure (ISO 37) to those obtained from a virtual coun-

terpart of the same experiment. As can be seen from Fig. 4, we

are able to match the real material behavior quite closely for defor-

mations in the range of [−20%, +50%]. Considering the operating

range of our animatronics device, this provides us with sufﬁciently

large safety margins.

5 Physical Simulation of Synthetic Skin

As a central part of the face cloning pipeline, we need to have an

accurate computational model for simulating deformations of a syn-

thetic skin. Our approach is to model skin as a hyperelastic isotropic

solid. Since we have to account for large rotations, stretching, and

compression, we resort to ﬁnite-deformation continuum mechan-

ics. Here we only give a brief overview and refer the reader to the

textbook by Bonet and Wood [?] for a comprehensive exposition.

Let X and x denote smooth functions describing the position of

the skin in its undeformed and deformed state, respectively. Fur-

ther, let ϕ : Ω → IR

3

denote the mapping that transforms mate-

rial points from the undeformed to the deformed conﬁguration as

Figure 3: Illustration of thickness optimization. (a) Target surface

(b) undeformed skin patch (c) unoptimized skin patch deformed by

vertical load (d) deformed skin patch with optimized thickness.

x = ϕ(X). The deformation of the skin at each point can be char-

acterized by the deformation gradient F or, alternatively, the right

Cauchy-Green tensor C which are deﬁned as

F =

∂ϕ

∂X

, respectively C = F

T

F .

For a hyperelastic material, the energy of the solid depends only on

its current state of deformation, which is described in a rotation-

and frame-invariant manner through the tensor C. We denote the

corresponding energy density function by Ψ = Ψ(C). The rela-

tion between Ψ and C is described by material models, which are

discussed next.

5.1 Material Model

Animating expressions on a virtual face generally induces both ﬁ-

nite stretching and compression of the skin. While the simple St.

Venant-Kirchhoff solid cannot be used in this setting [?], the non-

linear neo-Hookean, Mooney-Rivlin, and Ogden material models

are well suited for rubber-like substances such as silicone. Being

the simplest representative of this class, the strain energy density of

the compressible neo-Hookean material is given as

Ψ(C) =

µ

2

(J

−

2

3

tr(C) −3) −

κ

2

(J −1)

2

, (1)

where J = det F and µ, κ are shear and bulk moduli, which are

related to the familiar Young’s modulus E and Poisson’s ratio ν as

E =

9µκ

3κ +µ

, respectively ν =

3κ −2µ

2(3κ +µ)

. (2)

The neo-Hookean material offers only two parameters, but for the

range of deformations considered here, we found this model to be

sufﬁciently accurate for obtaining good correspondence with the

measured data. Having settled upon this material model, we de-

termine the model parameters µ and κ for a given material sample

using the numerical-experimental technique described in the previ-

ous section. Fig. 4 compares the force-strain curve for a measured

and simulated material.

5.2 Discretization

The optimization process described in the next section requires the

minimization of the elastic energy subject to boundary conditions.

To this end, we discretize (1) in space using tetrahedral ﬁnite ele-

ments. Let xa ∈ IR

3

, 1 ≤ a ≤ n denote deformed positions of a

tetrahedral mesh. We will use the superscript e to identify quanti-

ties pertaining to a given tetrahedral element. In particuar, we let

X

e

∈ IR

12

and x

e

∈ IR

12

denote position vectors for a given ele-

ment in its undeformed and deformed state, respectively.

Figure 4: Force-strain curve of real (black) and simulated (red)

material.

We proceed by approximating the conﬁguration mapping as ϕ ≈

a

xaNa where Na are piece-wise linear basis functions associ-

ated with the nodes of the mesh. The discrete deformation gradient

for each element e is then expressed as

F

e

= F(x

e

, X

e

) =

4

a=1

x

e

a

_

∂N

e

a

∂X

_

T

, (3)

where we emphasize the fact that for each element F

e

is a function

of its position in both undeformed and deformed conﬁguration. All

other discrete quantities required for computing the elastic energy

of a deformed element, i.e. tr(C

e

) and J

e

, follow directly from

(3). Furthermore, we note that the deformation gradient is constant

within each element since N

e

a

are piece-wise linear functions. With

this relation established, the discrete elastic energy of a deformed

element W

e

can be expressed as

W

e

(x

e

, X

e

) =

_

Ω

e

Ψ(F

e

) = Ψ(F

e

)V

e

(X

e

) , (4)

where Ω

e

is the element’s parameter domain and V

e

its unde-

formed volume. From now on, let x = (x

T

1

, . . . , x

T

n

)

T

and

X = (X

T

1

, . . . , X

T

n

)

T

denote vectors containing all deformed and

undeformed positions respectively. Summing up all elemental con-

tributions, we obtain the total energy for a deformed conﬁguration

as

W(x, X) =

e

W

e

(x

e

, X

e

) , (5)

We can now use this energy in a static equilibrium problem in or-

der to compute the deformation of the skin in response to actuator

placements, which translate into a set of position constraints. The

deformed conﬁguration of the skin is then determined as the mini-

mum of the total energy

x = argmin

x

(W(x, X) +Wext(x)) , (6)

where Wext summarizes external contributions to the potential, e.g.

due to gravity. Requiring the gradient of the energy to vanish at

equilibrium leads to a set of nonlinear equations, which are solved

in the standard way using Newton iterations with incremental load-

ing and line search for convergence control.

By solving (6), we can simulate skin deformation in response to

arbitrary actuator placements.

6 Optimization

Recall that our goal is to fabricate a synthetic skin which, when

deformed by the underlying electromechanical base, matches the

target expressions as closely as possible. In order to achieve this

requirement, we optimize the shape of the skin as well as the actu-

ation parameters of the base. This process is subject to a number of

constraints. We cannot change the outside surface of the skin. We

also cannot modify the attachment points of the actuators. Further-

more, the motion of the actuators is physically restricted.

In order to achieve the best results given these constraints, we have

to expose as many degrees of freedom as possible to the optimiza-

tion process. To this end, we optimize with respect to two sets of

parameters. First, we optimize the actuation parameters of the an-

imatronic device for every target expression. Second, we change

the thickness of the skin by modifying its shape on the inside – we

are free to do so everywhere except at the attachment points and in

their immediate vicinity. Practically, this allows the skin to produce

bulges and wrinkles where they did not appear before. We start by

describing the general optimization framework and then how we

apply it to the individual parameter sets.

6.1 Generic Optimization Framework

Let the function

ˆ

W(ˆ x, p) denote the total energy of our physical

system, including internal deformation energies and external con-

tributions to the potential. p is a vector of generic parameters for

which we want to optimize. ˆ x contains a subset of the deformed

positions x of the physical simulation. All other values of x are

either constant (constrained) or computable from p. In the most

general case, both the undeformed and deformed positions of our

simulation mesh are allowed to depend on p, and we can write

ˆ

W(ˆ x, p) = W(x(ˆ x, p), X(p)) +Wext(x(ˆ x, p)). (7)

For ﬁxed parameter values p, the physical system will assume the

equilibrium state

ˆ xeq(p) = argmin

ˆ x

_

ˆ

W(ˆ x, p)

_

(8)

minimizing the total energy of the system, leading to the necessary

condition of a vanishing residual force

∂

ˆ

W

∂ˆ x

¸

¸

¸

¸

ˆ xeq,p

= 0. (9)

The goal of the optimization will be to ﬁnd the optimal parameter

values p such that the positions x(ˆ xeq(p), p) match a desired tar-

get conﬁguration as closely as possible. This is measured in terms

of a matching energy E

match

(x). We additionally regularize the

parameters using a regularization energy Ereg(p).

We solve this as a minimization problem with respect to ˆ x and p,

with the following objective function:

E(ˆ x, p) =

1

2

γ

¸

¸

¸

¸

¸

¸

¸

¸

∂

ˆ

W

∂ˆ x

¸

¸

¸

¸

ˆ x,p

¸

¸

¸

¸

¸

¸

¸

¸

2

+E

match

(x(ˆ x, p)) +Ereg(p) (10)

The ﬁrst term penalizes the violation of condition (9), ensuring with

a sufﬁciently high penalty γ that the resulting ˆ x is close to a physi-

cal solution with vanishing residual forces.

Matching Energy. The matching energy measures how well a

conﬁguration x is in agreement with what we want the object to

look like. To this end, we embed a set of points in the simu-

lation domain and deform them according to the mapping ϕ. In

our implementation, we use a subset of points from high-resolution

3D scans (Sec. 4). Due to our choice of basis functions, the de-

formed position of a point can be computed as a linear combination

of four nodal positions. We then compute the matching energy as

the squared distance between the deformed points and their desired

target positions q,

E

match

(x) =

1

2

||Sx −q||

2

(11)

where the matrix S contains the weights for the computation of the

deformed points from x.

Numerical Optimization. We minimize (10) using a Newton

method. For this we need to compute its ﬁrst and second deriva-

tives with respect to ˆ x and p. While these computations are trivial

for our particular choices of matching and regularization energies,

we compute the second derivatives of the ﬁrst term in (10) only ap-

proximately, ignoring the third order derivatives of W that would

result from applying the chain rule. In each Newton step, we use a

sparse direct solver [?] to solve the linear system

H

_

∆ˆ x

∆p

_

= −f (12)

for the increments ∆ˆ x and ∆p, where f is the vector of ﬁrst deriva-

tives of E and H contains its second derivatives. We employ line

search to make sure the energy decreases in each Newton step. It

can happen that the matrix H becomes indeﬁnite, a condition we

can easily detect by the direct solver not being able to factorize H,

in which case we proceed as suggested in Nocedal and Wright [?]

by adding a multiple of the identity to H.

6.2 Thickness Optimization

The goal of the thickness optimization is to modify the local thick-

ness of the skin in such way that when the mechanical actuators of

the robot are set to the values corresponding to a particular pose,

the resulting deformation of the skin matches the pose’s target po-

sitions q as closely as possible. In the physical simulation, the actu-

ator settings result in hard positional constraints that can directly be

applied to the corresponding deformed positions. The parameters

p

thk

determine the thickness distribution in the undeformed conﬁg-

uration without directly affecting the deformed positions, thus we

can write

ˆ

W(ˆ x, p

thk

) = W(ˆ x, X(p

thk

)) +Wext(ˆ x) (13)

and minimize (10) to ﬁnd the parameter values for the thickness

distribution that best match the given pose.

Parameterizing the Undeformed State. Although the skin

clearly exhibits a thickness direction, this information is not explic-

itly available from the 3D geometry such that we have to infer it

ﬁrst. We start by computing a low-pass ﬁltered version of the skin’s

outer surface and construct a parameterization r : IR

2

→IR

3

using

standard uv-mapping software. A roughly area-preserving map is

desirable for obtaining unbiased sample distributions, but this is not

a critical aspect in this context. The undeformed geometry of the

skin can now be described as X = r(u, v)+hn(u, v), where u and

v are surface coordinates and the thickness parameter h runs along

the surface’s normal direction n. Having deﬁned the thickness in

this way, we can now proceed to its optimization.

Instead of working directly on the undeformed positions, we deﬁne

a warping heightﬁeld on the surface in order to smoothly deform

space along its normal direction. We construct a smooth warp-

ing ﬁeld w(u, v) from a sparse set of height samples αi at po-

sitions (ui, vi)

T

using moving least squares (MLS) interpolation

[?]. Fig. 5 illustrates the concepts of parametrization and MLS in-

terpolation, while Fig. 6 shows some shape variations that can be

Figure 5: Left: parametrization of the geometry and placement

of MLS samples; Right: MLS warping ﬁeld shown as surface with

isoparametric lines for illustration.

obtained for a simple one-dimensional example. To control the res-

olution of the warping heightﬁeld, we adaptively place the locations

of the height samples in uv space based on a user-provided density

map [?].

Evaluating the MLS warping ﬁeld at an arbitrary parameter point

(u, v) amounts to ﬁtting an afﬁne transformation that minimizes

the weighted least squares error

i

θ(||(u, v)

T

−(ui, vi)

T

||)||a(u, v)

T

(1, ui, vi) −αi||

2

, (14)

where θ(x) is a weighting kernel and a(u, v) ∈ IR

3

are the local

coefﬁcients of the interpolation function. The latter are determined

by solving the normal equations for (14), which can be done efﬁ-

ciently using precomputations.

Finally, we gather the parameters αi of all sample points into p

thk

and compute optimal height values by minimizing (10). Note

that the MLS interpolation results in a linear mapping between

undeformed positions X and parameters p

thk

, thus the matrix

∂X/∂p

thk

is constant and can be precomputed. Once the opti-

mal αi are determined, the ﬁnal rest conﬁguration is obtained in

the following way: for each point Xa of the original mesh we ﬁrst

retrieve its parameter values (ua, va, ha), evaluate the MLS warp-

ing ﬁeld to obtain wa = w(ua, va) and ﬁnally compute the warped

positions as X

a

= r(ua, va) +wahan(ua, va).

Regularization. We need the undeformed mesh with nodal posi-

tions X(p

thk

) to be a valid tetrahedral mesh where every element

has positive volume. While there are different ways to formulate

and enforce this constraint, we found it most practical to reuse the

deformation energy function W, abusing X(p

thk

) as the deformed

conﬁguration and the initial undeformed positions X(1) as the un-

deformed conﬁguration:

Ereg(p

thk

) = γ

undefreg

W(X(p

thk

), X(1)) (15)

For any positive penalty value γ

undefreg

, this energy will tend to

inﬁnity as any element’s volume approaches zero, effectively pre-

venting element inversions in the undeformed conﬁguration.

Fabrication Constraints. Additionally, we add a quadratic en-

ergy term to Ereg that penalizes MLS weights exceeding a value

of one, or falling below a user-deﬁned minimum thickness. This

guarantees a fabricatable skin that does not intersect the underlying

mechanical structure.

Multiple Pose Optimization. We want to optimize the unde-

formed conﬁguration X(p

thk

) such that m > 1 poses can be re-

produced as closely as possible: for a set of actuator settings ai, the

given target positions qi should be matched as closely as possible.

In this case our optimization is slightly modiﬁed and we optimize

for all deformed positions ˆ x1, . . . , ˆ xm and p

thk

at once. We have

Figure 6: Illustration of MLS interpolation using 7 sample points.

(a) Input geometry (b)–(d) examples of warped geometry.

m matching functions E

i

match

and the objective function can be

written as

E(ˆ x, p

thk

) =

m

i=1

1

2

γ

¸

¸

¸

¸

¸

¸

¸

¸

∂

ˆ

W

∂ˆ x

¸

¸

¸

¸

ˆ x

i

,p

thk

¸

¸

¸

¸

¸

¸

¸

¸

2

(16)

+

m

i=1

E

i

match

(x(ˆ xi, p

thk

)) (17)

+Ereg(p

thk

). (18)

We initialize the optimization with values ˆ xi computed from a per-

pose forward simulation with incremental loading, and use p

thk

=

1 for the MLS parameters.

6.3 Actuation Parameter Optimization

The goal of the actuation parameter optimization is to ﬁnd the best

values for controlling the mechanical actuators such that the result-

ing deformed skin matches a given target pose as closely as possi-

ble. In this setting, the parameters pact of the optimization deﬁne

the positional constraints to apply to the nodal positions x. In the

objective function (10) we use

ˆ

W(ˆ x, pact) = W(x(ˆ x, pact), X) +Wext(x(ˆ x, pact)). (19)

Parameterization. The mapping from unconstrained nodal posi-

tions ˆ x and parameters pact to deformed positions x is

xi =

_

ˆ xi if node i unconstrained

Ma(pact)Xi if node i constrained by actuator a.

(20)

For the sake of simplicity, we linearize the transformation matrix

Ma(pact) of actuator a around the current parameter values pact

in each step of the optimization.

Regularization. Similar to the thickness optimization, we use the

regularization energy Ereg to keep the actuator parameter values

within the range of physically feasible values.

7 Animatronic Head

To actuate the silicone skin we use a proprietary ani-

matronic head developed by Walt Disney Imagineering.

It is driven by electric motors and features

a = 13 parameters pact ∈ IR

a

to con-

trol the movement of the attachment links

indicated in red in the ﬁgure on the left.

The length of the red arrows illustrates

the range of potential motion. Further-

more, there are several rigid attachment

links that constrain the movement of the

skin. We refer to the video for a visual-

ization of the actuation level of each ac-

tuation parameter for a facial animation

sequence, providing an intuition of which

expressions are inside or at the border of

the gamut of reproducible poses. We treat the underlying mechan-

ics and controlling mechanism as a black box, so in theory our op-

timization algorithm can be used with any animatronics device.

To determine the operating range and control parameters of the head

we attach a marker to each moving link and track its position under

a number of sample locations and orientations. Tracking is per-

formed using the same capture system as used for face scanning

(Sec. 4). The mapping m(pact) : IR

a

→ IR

l·6

from animation pa-

rameters pact to 3D locations and orientations of attachment points

is then speciﬁed by quadratically interpolating the samples. These

are then used as hard boundary constraints in our forward simula-

tion. To obtain the speciﬁc animation parameters for a given facial

expression, we solve the inverse problem as described in Sec. 6.3.

8 Fabrication

Our choice to use silicone is based on the fact that the stiffness can

be controlled accurately, the color can be adjusted, it is robust, and

the fabrication process is easy and safe. Silicone belongs to the

class of elastically deformable polymers with a high yield strength.

After having optimized the ideal skin geometry, we use liquid injec-

tion molding to fabricate the physical skin as shown in the accom-

panying video. Our mold consists of a core that resembles the inner

surface of the skin and six outer parts. Note that multiple outer parts

are required so that the skin can be removed without destroying the

mold. The mold itself is fabricated using a rapid prototyping 3D

printer. For our experiments, we used the two-part silicone GI-245

by Silicones Inc. After mixing it with the corresponding catalyst, it

has a liquid consistency and can be injected into the mold. Approx-

imately seven days of room-temperature curing is required until it

is ready to be used.

9 Results

In this section we present our results, starting with validation of our

pipeline and optimization process, and then showcasing our method

with an example of real physical face cloning for an animatronic

ﬁgure.

Validation Simulation. We validated our pipeline for measuring,

modeling, and simulating silicone materials. Fig. 4 shows a com-

parison of the force-strain curve of a simulated material to a mea-

sured sample. The measurements are acquired by running an uni-

axial stretch and compression test on a rectangular (55 × 5 × 2

mm — stretch) and cylindrical (50 mm diameter, 12 mm height

— compression) object. Furthermore, we validate the simulation

by comparing simulated deformations of our silicone skin with 3D

scanned poses of the actual animatronic ﬁgure in Fig. 9, showing

an average error of less than 1.35 mm.

Validation Optimization. Our optimization process computes

the volumetric shape of soft-tissue material under actuation such

that the surface matches the deformation of given target surfaces.

We validated the optimization process in simulation. We start by

generating ground truth data by randomly varying the thickness of

a uniformblock using the moving least squares warp as described in

Sec. 6.2. From the forward simulation of these objects we extract

target surfaces. We then used these target surfaces and the unde-

formed block as input into our optimization — see e.g. Fig. 7. In

(a) (b) (c) (d) (e)

Figure 7: Volumetric shape optimization. Starting with a volumet-

ric object (a) and a target surface (b), our optimization process

computes the thickness variation (c) such that the surface matches

the deformation of the target under actuation (d). Without optimiza-

tion (e), the target is not well approximated.

all experiments, our algorithm was able to exactly reproduce the

spatially-varying thickness of the ground truth objects that were

used to generate the inputs. As shown in Fig. 8, we also fabricated

a subset of these examples, and compared the deformation of the

predicted optimized shape to the real object. For these 95mm long

bars, the average errors between the deformed fabricated surface

and the goal shape are 0.44mm (top) and 0.38mm (bottom).

Face Cloning. We showcase our complete pipeline by physically

cloning a human face for an animatronic ﬁgure. We started by ac-

quiring more than 100 frames of an expressive performance of our

subject. We then aligned the neutral pose to a scan of the anima-

tronic head. Because the animatronic head has very pronounced

cheek bones, we slightly warp the acquired 3D geometry of the

human face to prevent intersections. Consequently, we apply the

same warping transformation to the captured geometry before do-

ing comparisons (such as the error plots in Fig. 10) with our results.

The gap between the surface of the neutral pose and the underlying

animatronics deﬁnes the initial unoptimized skin, which we repre-

sent as a tetrahedral mesh consisting of ≈ 63k elements. To better

represent important facial deformation features on the forehead, we

additionally use a high-resolution mesh of ≈ 157k elements to op-

timize this area. We then use our optimization process to obtain

animation parameters for the entire captured face performance se-

quence. As matching criteria we select small surface regions close

to the attachment links. In those regions, the surface thickness is

constrained and cannot be changed because the skin is connected

to the moving and static links. For all other regions we use our op-

timization process to achieve the desired deformation behavior by

varying the skin thickness. In our experiment, we used a subset of 8

expressive poses for thickness optimization. Fig. 10 illustrates the

unoptimized and optimized geometry and provides a side-by-side

comparison of the deformation behavior.

Finally, we also fabricated the optimized skin, using material with

Young’s modulus of E = 78kPa and Poisson’s ratio of ν = 0.47

in its cured state. Figure ﬁnishing included industry-standard tech-

niques for adding hair and facial stubbles and painting a texture

by a professional makeup artist. Fig. 9 shows our ﬁgure in action

and compares it to the human face and the simulation under vari-

ous facial expressions. We also scanned these poses of the anima-

tronic ﬁgure and provide error visualizations showing the distance

between the simulated surface and the scanned surface. As can be

seen from the error plots, the agreement between the shape pre-

dicted by our simulation and the actual observed shape of the actu-

ated skin is generally good, with less than 4mm separating the sur-

faces everywhere. This shows that our simulation is a good model

for the real synthetic face. The difference between the scans of the

real face and the simulation is larger, representing the limitations in

the range of motion of the robot head and the range of deformation

of the attached skin. Comparing the human face to the cloned syn-

thetic face, the large scale deformations and general expressions

are captured well, and the differences show where improvements

should be made in the physical system to enable the ﬁtting methods

to achieve a better match. For further comparisons we refer to the

accompanied video.

10 Conclusion

In this paper we have described a process for computationally

guided design of an animatronic character composed of a soft-tissue

material and an electromechanical base. We compute a volumetric

shape of the soft-tissue material and a set of actuation forces that

can reproduce target deformations of an exterior character surface.

As an example, this process allows us to construct an animatronic

head that approximates the shape and behavior of a real person

whose expressions are acquired using a 3D scanning system. The

core requirement of this process is a physically based simulation

framework that lets us accurately predict the behavior of the soft

tissue when subject to external forces. Our experimental validation

shows that using this process provides a principled way to design

soft-tissue animatronic characters.

Limitations and Future Work. We believe that our computation-

ally guided process for designing and fabricating animatronic char-

acters will serve as a blueprint for how future robots with soft tissue

should be built. Because this work is the ﬁrst in the area, it has a

number of limitations that should be addressed in future research.

First, in our current framework the attachment of the actuators is

ﬁxed to speciﬁc places underneath the character’s skin. Relaxing

this constraint would likely give us more degrees of freedom, e.g.,

the rest shape of the soft tissue would not need to be the same as

the rest shape of the electromechanical base. Second, we currently

only use a single soft tissue material to produce a synthetic skin. We

believe we can extend the range of possible deformations by using

multi-layered materials similarly to Bickel et al. [?]. Moreover, it

is important to note that the motions of current animatronic charac-

ters are still limited as they are less expressive than most humans.

Thus, it is important to develop retargeting algorithms that can map

the expressions of a target human to the physical constraints of the

electromechanical base and materials used. In this paper, we have

not addressed the appearance aspect of the synthetic skin tissue in-

cluding reﬂectance, subsurface scattering, and hair. This is also a

very interesting direction for future work and we envision that sim-

ilar computationally guided processes could be developed to solve

these problems. Finally, while in this work we have only shown

examples of replicating human faces, we predict that similar frame-

works will be used to design realistic full-body characters.

Acknowledgements

We would like to thank the anonymous reviewers for their com-

ments, Brian Tye, Frank Mezzatesta, and Walt Disney Imagineering

for their support, and Lee Romaire for ﬁgure ﬁnishing. This work

was supported in part by the NCCR Co-Me of the Swiss NSF.

References

BECKER, M., AND TESCHNER, M. 2007. Robust and efﬁcient

estimation of elasticity parameters using the linear ﬁnite element

method. In SimVis, 15–28.

BEELER, T., BICKEL, B., BEARDSLEY, P., SUMNER, B., AND

GROSS, M. 2010. High-quality single-shot capture of facial

geometry. ACM Trans. Graph. 29, 4 (July), 40:1–40:9.

BEELER, T., HAHN, F., BRADLEY, D., BICKEL, B., BEARDS-

LEY, P., GOTSMAN, C., SUMNER, R. W., AND GROSS, M.

2011. High-quality passive facial performance capture using an-

chor frames. ACM Trans. Graph. 30, 4 (August), 75:1–75:10.

BICKEL, B., BOTSCH, M., ANGST, R., MATUSIK, W., OTADUY,

M., PFISTER, H., AND GROSS, M. 2007. Multi-scale capture

of facial geometry and motion. ACM Trans. Graph. 26, 3 (July),

33:1–33:10.

BICKEL, B., B¨ ACHER, M., OTADUY, M. A., MATUSIK, W.,

PFISTER, H., AND GROSS, M. 2009. Capture and modeling

of non-linear heterogeneous soft tissue. ACM Trans. Graph. 28,

3 (July), 89:1–89:9.

BICKEL, B., B¨ ACHER, M., OTADUY, M. A., LEE, H. R., PFIS-

TER, H., GROSS, M., AND MATUSIK, W. 2010. Design and

fabrication of materials with desired deformation behavior. ACM

Trans. Graph. 29, 4 (July), 63:1–63:10.

BONET, J., AND WOOD, R. D. 1997. Nonlinear Continuum Me-

chanics for Finite Element Analysis. Cambridge Uni. Press.

BRADLEY, D., BOUBEKEUR, T., AND HEIDRICH, W. 2008. Ac-

curate multi-view reconstruction using robust binocular stereo

and surface meshing. In Proc. CVPR.

BRADLEY, D., HEIDRICH, W., POPA, T., AND SHEFFER, A.

2010. High resolution passive facial performance capture. ACM

Trans. Graph. 29, 4 (July), 41:1–41:10.

BUCUR, D., AND BUTTAZZO, G. 2005. Variational Methods in

Shape Optimization Problems. Birkhuser Mathematics.

CHENTANEZ, N., ALTEROVITZ, R., RITCHIE, D., CHO, L.,

HAUSER, K. K., GOLDBERG, K., SHEWCHUK, J. R., AND

O’BRIEN, J. F. 2009. Interactive simulation of surgical needle

insertion and steering. ACM Trans. Graph. 28, 3 (July), 88:1–

88:10.

CLARBERG, P., JAROSZ, W., AKENINE-M¨ OLLER, T., AND

JENSEN, H. W. 2005. Wavelet importance sampling: efﬁciently

evaluating products of complex functions. ACM Trans. Graph.

24, 3 (Aug.), 1166–1175.

DONG, Y., WANG, J., PELLACINI, F., TONG, X., AND GUO, B.

2010. Fabricating spatially-varying subsurface scattering. ACM

Trans. Graph. 29 (July), 62:1–62:10.

GOURRET, J.-P., THALMANN, N. M., AND THALMANN, D.

1989. Simulation of object and human skin deformations in a

grasping task. In Comp. Graph. (Proc. SIGGRAPH), 21–30.

HARA, F., AKAZAWA, H., AND KOBAYASHI, H. 2001. Realistic

facial expressions by sma driven face robot. In Proc. of IEEE

International Workshop on Robot and Human Interactive Com-

munication, 504 –511.

HAˇ SAN, M., FUCHS, M., MATUSIK, W., PFISTER, H., AND

RUSINKIEWICZ, S. 2010. Physical reproduction of materi-

als with speciﬁed subsurface scattering. ACM Trans. Graph. 29

(July), 61:1–61:10.

IRVING, G., TERAN, J., AND FEDKIW, R. 2004. Invertible ﬁnite

elements for robust simulation of large deformation. In 2004

ACM SIGGRAPH / Eurographics SCA, 131–140.

KAUER, M., VUSKOVIC, V., DUAL, J., SZEKELY, G., AND BA-

JKA, M. 2002. Inverse ﬁnite element characterization of soft

tissues. Medical Image Analysis 6, 3, 257–287.

(a) (b) (c) (d) (e)

Figure 8: Illustrating the volumetric shape optimization in 1D. Given a volumetric object and a target surface (indicated in orange) as shown

in column (a), our optimization process computes the volumetric shape (b) such that the surface matches the deformation of the target under

actuation (c). Columns (d) and (e) show the undeformed and deformed state of the corresponding fabricated objects against a 4mm grid.

Figure 9: Side-by-side comparison. From left to right: Photographs of captured facial expressions; Renderings of 3D reconstructions; error

comparison of 3D reconstructions to the fabricated skin (blue 0mm, red 4mm); 3D simulation of the animatronic head for the corresponding

pose; error comparison of our simulation to the fabricated skin (blue 0mm, red 4mm); expressions generated with the optimized fabricated

skin actuated by an animatronic device.

(a) (c) (e) (g)

(b) (d) (f) (h)

Figure 10: Forehead Optimization: (a) Rest shape. (b) Target geometry. (b,c) Rendering of the inner side of the unoptimized and optimized

forehead with color-coded thickness variation. Comparison of the unoptimized and optimized skin: Error plot of the (e) unoptimized and (f)

optimized skin (blue 0mm, red 4mm). Although the variations are subtle, a comparison of the fabricated unoptimized (g) and optimized (h)

skin shows visible differences.

KHAREVYCH, L., MULLEN, P., OWHADI, H., AND DESBRUN,

M. 2009. Numerical coarsening of inhomogeneous elastic ma-

terials. ACM Trans. Graph. 28 (July), 51:1–51:8.

KOCH, R. M., GROSS, M. H., CARLS, F. R., VON B¨ UREN, D. F.,

FANKHAUSER, G., AND PARISH, Y. I. H. 1996. Simulating

facial surgery using ﬁnite element models. In Proc. of Comp.

graph. and int. tech., ACM, SIGGRAPH ’96, 421–428.

LEE, S.-H., AND TERZOPOULOS, D. 2006. Heads up!: biome-

chanical modeling and neuromuscular control of the neck. In

ACM SIGGRAPH 2006 Papers, ACM, 1188–1198.

LEE, S.-H., SIFAKIS, E., AND TERZOPOULOS, D. 2009. Com-

prehensive biomechanical modeling and simulation of the upper

body. ACM Trans. Graph. 28 (September), 99:1–99:17.

LEVIN, D. 1998. The approximation power of moving least-

squares. Math. Comput. 67 (October), 1517–1531.

LI, X.-Y., SHEN, C.-H., HUANG, S.-S., JU, T., AND HU, S.-M.

2010. Popup: automatic paper architectures from 3d models.

ACM Trans. Graph. 29 (July), 111:1–111:9.

MORI, Y., AND IGARASHI, T. 2007. Plushie: an interactive design

system for plush toys. In ACM SIGGRAPH 2007 papers, ACM,

SIGGRAPH ’07.

NESME, M., KRY, P. G., JE ˇ R ´ ABKOV´ A, L., AND FAURE, F. 2009.

Preserving topology and elasticity for embedded deformable

models. In ACM SIGGRAPH 2009 papers, ACM, 52:1–52:9.

NISHIO, S., ISHIGURO, H., AND HAGITA, N. 2007. Humanoid

Robots: NewDevelopments. I-Tech, ch. Geminoid: Teleoperated

Android of an Existing Person.

NOCEDAL, J., AND WRIGHT, S. J. 2000. Numerical Optimization.

Springer, August.

OH, J.-H., HANSON, D., KIM, W.-S., HAN, I. Y., KIM, J.-Y.,

AND PARK, I.-W. 2006. Design of android type humanoid robot

albert hubo. In Proc. of IEEE/RSJ Int. Conference on Intelligent

Robots and Systems, 1428–1433.

PAI, D. K., DOEL, K. V. D., JAMES, D. L., LANG, J., LLOYD,

J. E., RICHMOND, J. L., AND YAU, S. H. 2001. Scanning

physical interaction behavior of 3d objects. In Proc. of Comp.

graph. and int. tech., ACM, SIGGRAPH ’01, 87–96.

SCHENK, O., AND G¨ ARTNER, K. 2006. On fast factorization piv-

oting methods for symmetric indeﬁnite systems. In Elec. Trans.

Numer., no. 23, 158–179.

SIFAKIS, E., NEVEROV, I., AND FEDKIW, R. 2005. Automatic

determination of facial muscle activations from sparse motion

capture marker data. In ACM SIGGRAPH 2005, 417–425.

SUEDA, S., KAUFMAN, A., AND PAI, D. K. 2008. Musculoten-

don simulation for hand animation. ACM Trans. Graph. (Proc.

SIGGRAPH) 27, 3.

TERAN, J., SIFAKIS, E., BLEMKER, S. S., NG-THOW-HING, V.,

LAU, C., AND FEDKIW, R. 2005. Creating and simulating

skeletal muscle from the visible human data set. IEEE Trans. on

Vis. and Comp. Graph. 11, 3, 317–328.

TERZOPOULOS, D., AND WATERS, K. 1993. Analysis and syn-

thesis of facial image sequences using physical and anatomical

models. IEEE Trans. Pattern Anal. Mach. Intell. 15 (June), 569–

579.

TERZOPOULOS, D., PLATT, J., BARR, A., AND FLEISCHER, K.

1987. Elastically deformable models. In Proc. Comp. graph.

and int. tech., ACM, SIGGRAPH ’87, 205–214.

THOUTIREDDY, P., AND ORTIZ, M. 2004. Avariational r-adaption

and shape-optimization method for ﬁnite-deformation elasticity.

Int. J. Numer. Meth. Engng. 61, 1–21.

VAN GELDER, A. 1998. Approximate simulation of elastic mem-

branes by triangulated spring meshes. J. Graph. Tools 3, 2, 21–

42.

WANG, Y., HUANG, X., LEE, C.-S., ZHANG, S., LI, Z., SAMA-

RAS, D., METAXAS, D., ELGAMMAL, A., AND HUANG, P.

2004. High resolution acquisition, learning and transfer of dyan-

mic 3-d facial expressions. In Comp. Graph. Forum, 677–686.

WEYRICH, T., PEERS, P., MATUSIK, W., AND RUSINKIEWICZ,

S. 2009. Fabricating microgeometry for custom surface re-

ﬂectance. ACM Trans. Graph. 28 (July), 32:1–32:6.

ZHANG, L., SNAVELY, N., CURLESS, B., AND SEITZ, S. M.

2004. Spacetime faces: high resolution capture for modeling

and animation. ACM Transactions on Graphics 23, 3 (Aug.),

548–558.

In the context of computer graphics. ?. we use a dedicated parametrization approach combined with a moving least squares interpolation technique that allows us to modify the shape only in its thickness direction. has been carried out by Intelligent Research Laboratory at Osaka University. In contrast to our approach. we fabricate synthetic skins with uniform material properties and optimize their shape. ?. which are adequately modeled as elastic materials with isotropic and homogeneous properties. design. the design of reﬂectors for luminaires or the optimization of material interfaces. [?] was one of the ﬁrst group to build an anthropomorphic head to study human-robot interaction. Hara et al. [?] describe a comprehensive system for capturing interactions with real-world objects. fabrication-oriented design is a recent trend that is seeing increasing attention in computer graphics. Robotics & Animatronics. There exists a multitude of methods for capturing facial expressions. nonlinear viscoelastic soft tissue [?]. the work by Teran et al. including deformation response. scared. • A comprehensive analysis and experimental validation of the process and its individual steps. ?]. surface roughness and contact sounds. There exists a plethora of dedicated bio-mechanical models for different parts of the human body such as the face [?. The head was able to reproduce six different expressions such as surprised. fabrication and validation. targeting the reproduction of geometric features of papercraft [?] or plush toys [?]. Measuring and Fitting Material Parameters. Such problems are typically cast as constrained optimization problems and solved numerically with ﬁnite elements (see. 4). The group has built a number of robotic characters with increasing capabilities. 3 Overview Our approach to physical face cloning comprises modeling. including data acquisition.g. ?]. Several methods have been presented in this context. as in our case. disgusted. . But since it is generally difﬁcult to control the adaptation of the undeformed conﬁguration. As a natural step to follow acquisition and data-driven simulation. In addition. Our approach for determining parameters of different silicone samples goes along similar lines: we use a vision-based acquisition system to capture dense surface displacement ﬁelds and determine optimal material parameters by minimizing the displacement error of a corresponding ﬁnite element approximation. including traditional marker-based motion capture [?. Android research. The individual steps of our processing pipeline are illustrated in Fig. We optimize the shape of the synthetic skin by minimizing an elastic energy with respect to rest state positions. angry. structured light systems [?. and material sciences. Similarly. The robot’s head resembles Albert Einstein and its underlying electro-mechanical actuation has 35 degrees of freedom. We start by capturing facial expressions of a human subject using an optical performance capture system that offers high-resolution 3D reconstructions including pores and wrinkles. Simulation. Examples of such shape optimization problems include the computation of aerodynamic proﬁles. ?. on the computational side. the hand [?. Their recent humanoid robot Geminoid HI-1 [?] as well as it’s successor Geminoid-DK have the realistic appearance of an adult man. several methods have been described for ﬁtting measured data to computational models. The face skin was made of silicone and it was driven by SMA (shape memory alloy) actuators. which is similar in spirit to methods for variational shape adaptation [?]. optimization. the textbook by Bucur and Buttazzo [?] for an introduction). the individual components of our system have diverse connections to previous research. we rely on a compressible neo-Hookean material model and. e. 2. and passive markerless approaches [?. simulation. it has become common to model elastically deformable materials using continuum mechanics and ﬁnite elements. employ a ﬁnite element approach centered around linear tetrahedral elements. e. applied mathematics. Our acquisition system uses the recent method of Beeler et al. However. Physical face cloning is a complex task that intersects with numerous research areas from computer graphics. Introduced to the graphics community by Terzopoulos et al [?]. 2 Related Work Engineering applications often require the shape of a structure to be optimal with respect to given conditions. the design process for all these robotic skins is mainly manual. synthesizing and transferring facial expressions are important research topics in computer graphics. or even the entire upper body [?]. Face Capture. from measurements of realworld objects. The majority of robots consist of only rigid mechanical structures. These robots also have silicone-based skin driven by actuators placed underneath. [?] and references therein). and the implications of changes to the shape have to be tested in a time consuming trial and error manner. This reference data can originate either from an accurate computational model [?. Shape variations are modeled by an artist. Our approach for fabricationoriented material design bears some similarities to this last reference. Shape Optimization. physical simulation. a large body of literature is concerned with modeling of biological soft tissue (see. Although we are not aware of existing work to address this problem in its entirety.g. determine spatially varying material properties for ﬁxed geometry. but rather take an output-oriented approach: we fabricate synthetic skin using silicone rubbers.. The skin is made of a sponge-like elastomer which is very soft and requires very little force to deform. ?]. Owing to the expressive power of the human face and our ability to infer a burst of information from the slightest motion. or nonlinear heterogeneous materials [?]. as well as robust temporal correspondence (Sec. KAIST and Hanson Robotics have developed a robot called Albert HUBO [?]. [?] to capture short performance sequences with high-resolution geometry and dense temporal correspondences. We do not strive to reproduce the mechanical properties of biological tissue. ?]. ?]. ?]. Pai et al. general appearance properties [?. ?. including linear-elastic materials [?]. ?] or. but there is also a crucial difference: whereas Bickel et al. • A complete process for automating the physical reproduction of a human face on an animatronics device. This data provides information about the deformation behavior of the subject’s skin.• A method to optimize for synthetic skin geometry and actuation parameters that provide the best match to a collection of given target expressions. and also the elastic properties of deformable media [?]. robots that are covered with soft-tissue materials such as rubbers or silicones have been built as well. various organs [?]. Closely related to the choice of a constitutive law is the problem of ﬁnding a set of parameters that best approximates a given reference model. which involves studying human-computer interaction.. sad and happy. More speciﬁcally. Fabrication-oriented Design. capturing. and fabrication of synthetic soft tissue. the torso [?].

The vector p holds the physical parameters of the material model. evaluate the objective function O(pi ) and determine updated parameters pi+1 . The measurement proceeds by pulling on small samples with controlled forces and capturing the resulting deformation over time using stereo reconstruction [?]. Silicone rubber is an ideal base material for fabricating synthetic skin since it is easy to process and offers a vast range of stretch resistance. All these degrees of freedom are handled in one uniform optimization framework (Sec. we also determine the operational range of the animatronic head. compatible mesh sequence with explicit temporal correspondence. Our approach is to capture a short performance of the target subject. Hence. we ﬁrst measure the force-deformation behavior of a collection of silicone samples with different amounts of plasticizer. let ϕ : Ω → IR3 denote the mapping that transforms material points from the undeformed to the deformed conﬁguration as We rely on real-world data for modeling facial geometry and expressions as well as material behavior. Measuring & Fitting Material Parameters. we also optimize the actuation parameters of the animatronic device to ﬁnd the parameters that best resemble the individual target expressions. we numerically ﬁt our computational model to experimentally acquired forcedisplacement samples of materials with different stiffnesses. The following paragraphs brieﬂy describe our techniques for data acquisition and integration. we optimize the material parameters of our ﬁnite element solver in order to best approximate the measured stress-strain behavior. The result is a detailed. . To this end. Further. Considering the operating range of our animatronics device. we compare the stress-strain curves obtained from a standard testing procedure (ISO 37) to those obtained from a virtual counterpart of the same experiment. we chose 8 expressive poses from the sequence for optimizing the skin parameters. Since we have to account for large rotations. With a set of applied loads and resulting surface displacements determined. In each iteration. we have to adjust the silicone composition such that the net stiffness of the synthetic skin complies to these constraints. stretching. +50%]. To accomplish this task. we are able to match the real material behavior quite closely for deformations in the range of [−20%. 9). Material Sample Material Parameters Actuation Parameters Final Skin Acquisition Optimization Fabrication Figure 2: Physical face cloning pipeline. Face Scanning. [?]. the goal is now to design synthetic soft tissue that matches the example shapes as closely as possible. which imposes an upper bound on the net stiffness of the synthetic skin. Here we only give a brief overview and refer the reader to the textbook by Bonet and Wood [?] for a comprehensive exposition. leveraging the high resolution geometry acquisition method of Beeler et al. While the outer surface of the skin in the undeformed conﬁguration is given by a 3D scan of the neutral target pose. 5 Physical Simulation of Synthetic Skin 4 Measurements and Acquisition As a central part of the face cloning pipeline. which allows us to sample a mapping of actuation parameters pact to 3D location and orientation of each link in space. We then compare predicted simulation results and real-world behavior for different deformation scenarios (Sec. we need to have an accurate computational model for simulating deformations of a synthetic skin. 8).Face Meshes f(pi)= Animatronic Head Skin Attachement Paths Material Thickness 3D Printed Mold When cloning a human face. 6). Our base material for the synthetic skin is silicone. and compression. In order to determine material parameters for the skin simulator. The molds are created using a rapid prototyping 3D printer. 4. As can be seen from Fig. which consists of a set of skin attachment links actuated by motors. which offers a wide range of stiffness that can be controlled by adjusting the concentration of plasticizer in the compound. Given an initial guess p0 we iteratively compute updated parameters in a simulated annealing process. The forward simulation allows accurate prediction of the deformed shape of a given synthetic skin and the resulting forces induced by the moving skin attachment links. Our approach is to model skin as a hyperelastic isotropic solid. this provides us with sufﬁciently large safety margins. matching the shape of the face and how the skin deforms under different expressions is one of the main challenges. Let X and x denote smooth functions describing the position of the skin in its undeformed and deformed state. we start by 2 deﬁning an objective function O(p) = x i | (˜ i − xi (p)) | that measures the difference between simulated surface positions xi (p) ˜ and their closest corresponding point of the captured surface xi . We then perform numerical optimization in order to determine the parameters of our computational model that best match the experimental data. We must reconstruct various 3D expressions of the subject to be cloned. We attach to each moving link a marker. respectively. In our case. 5). Interpolating these samples yields a continuous function m(pact ) that describes the path of the skin attachment links (Sec. we resort to ﬁnite-deformation continuum mechanics. 5). Furthermore. The animatronics device can only exert a limited force through its actuators. We model and simulate the deformation behavior using a non-linear ﬁnite element method in combination with a neo-Hookean material (Sec. This process is repeated until the objective function signals convergence. we compute the ﬁnite element solution x(pi ) with the current parameters. in order to seed the deformable skin optimization. In order to design a synthetic skin that best matches a desired target. 7). which can be identiﬁed with Young’s modulus and Poisson’s ratio (see Sec. As a validation for this numerical-experimental material ﬁtting process. Given the acquired example shapes of the human face and the description of the animatronic head. we propose to vary the inner surface (thickness) of the skin to achieve spatially varying stiffness and deformation behavior. we introduce a novel optimization process. We validate our simulation and optimization approach by fabricating various silicone objects using injection molding (Sec. allowing us to analyze the deformation of a face under various expressions. down to the level of individual wrinkles.

Figure 4: Force-strain curve of real (black) and simulated (red) material. xT )T and 1 n X = (XT . Having settled upon this material model. i. we note that the deformation gradient is constant e within each element since Na are piece-wise linear functions. The discrete deformation gradient for each element e is then expressed as 4 Fe = F(xe . XT )T denote vectors containing all deformed and 1 n undeformed positions respectively. which are discussed next. x The neo-Hookean material offers only two parameters. (4) where Ωe is the element’s parameter domain and V e its undeformed volume. but for the range of deformations considered here. respectively ∂X C = FT F .e. we let Xe ∈ IR12 and xe ∈ IR12 denote position vectors for a given element in its undeformed and deformed state.Figure 3: Illustration of thickness optimization. Summing up all elemental contributions. we discretize (1) in space using tetrahedral ﬁnite elements. The deformed conﬁguration of the skin is then determined as the minimum of the total energy x = argmin (W (x. 2(3κ + µ) (2) We can now use this energy in a static equilibrium problem in order to compute the deformation of the skin in response to actuator placements. In particuar.2 Discretization The optimization process described in the next section requires the minimization of the elastic energy subject to boundary conditions. . which are related to the familiar Young’s modulus E and Poisson’s ratio ν as E= 9µκ . From now on. tr(Ce ) and J e . Mooney-Rivlin. (6) where Wext summarizes external contributions to the potential. 1 ≤ a ≤ n denote deformed positions of a tetrahedral mesh. Venant-Kirchhoff solid cannot be used in this setting [?]. Fig. . x = ϕ(X). . We proceed by approximating the conﬁguration mapping as ϕ ≈ a xa Na where Na are piece-wise linear basis functions associated with the nodes of the mesh. . By solving (6). and Ogden material models are well suited for rubber-like substances such as silicone. 6 Optimization Recall that our goal is to fabricate a synthetic skin which. Requiring the gradient of the energy to vanish at equilibrium leads to a set of nonlinear equations. which translate into a set of position constraints. . 5. matches the target expressions as closely as possible. follow directly from (3). e. which is described in a rotationand frame-invariant manner through the tensor C. we determine the model parameters µ and κ for a given material sample using the numerical-experimental technique described in the previous section. respectively. Let xa ∈ IR3 .1 Material Model where we emphasize the fact that for each element Fe is a function of its position in both undeformed and deformed conﬁguration. The deformation of the skin at each point can be characterized by the deformation gradient F or. which are solved in the standard way using Newton iterations with incremental loading and line search for convergence control. alternatively. X) + Wext (x)) . . Xe ) = Ωe Animating expressions on a virtual face generally induces both ﬁnite stretching and compression of the skin. when deformed by the underlying electromechanical base. 4 compares the force-strain curve for a measured and simulated material. the discrete elastic energy of a deformed element W e can be expressed as W e (xe . We will use the superscript e to identify quantities pertaining to a given tetrahedral element. X) = W e (xe . let x = (xT . 5. In order to achieve this requirement. (a) Target surface (b) undeformed skin patch (c) unoptimized skin patch deformed by vertical load (d) deformed skin patch with optimized thickness. we found this model to be sufﬁciently accurate for obtaining good correspondence with the measured data. Xe ) = a=1 xe a e ∂Na ∂X T . we obtain the total energy for a deformed conﬁguration as W (x. Furthermore. the nonlinear neo-Hookean. κ are shear and bulk moduli. This process is subject to a number of . the strain energy density of the compressible neo-Hookean material is given as Ψ(C) = κ µ −2 (J 3 tr(C) − 3) − (J − 1)2 . 2 2 (1) Ψ(Fe ) = Ψ(Fe )V e (Xe ) . respectively 3κ + µ ν= 3κ − 2µ . All other discrete quantities required for computing the elastic energy of a deformed element.g. (3) For a hyperelastic material. To this end. Being the simplest representative of this class. Xe ) . due to gravity. the energy of the solid depends only on its current state of deformation. we optimize the shape of the skin as well as the actuation parameters of the base. . we can simulate skin deformation in response to arbitrary actuator placements. We denote the corresponding energy density function by Ψ = Ψ(C). (5) e where J = det F and µ. While the simple St. The relation between Ψ and C is described by material models. With this relation established. . the right Cauchy-Green tensor C which are deﬁned as F= ∂ϕ .

X(p)) + Wext (x(ˆ. we embed a set of points in the simulation domain and deform them according to the mapping ϕ. p)). To this end. Furthermore. The undeformed geometry of the skin can now be described as X = r(u. p) = W (x(ˆ. we can now proceed to its optimization. We start by describing the general optimization framework and then how we apply it to the individual parameter sets. The matching energy measures how well a conﬁguration x is in agreement with what we want the object to look like. p) denote the total energy of our physical system. we change the thickness of the skin by modifying its shape on the inside – we are free to do so everywhere except at the attachment points and in their immediate vicinity. In each Newton step. Second. We additionally regularize the parameters using a regularization energy Ereg (p). In our implementation. The parameters pthk determine the thickness distribution in the undeformed conﬁguration without directly affecting the deformed positions. Having deﬁned the thickness in this way. p) = x ˆ 1 ∂W γ 2 ∂ˆ x 2 and minimize (10) to ﬁnd the parameter values for the thickness distribution that best match the given pose. Ematch (x) = 1 ||Sx − q||2 2 (11) where the matrix S contains the weights for the computation of the deformed points from x. we use a sparse direct solver [?] to solve the linear system Numerical Optimization. and we can write ˆ x W (ˆ. In the most general case. leading to the necessary condition of a vanishing residual force ˆ ∂W ∂ˆ x = 0. pthk ) = W (ˆ.p ˆ (9) The goal of the thickness optimization is to modify the local thickness of the skin in such way that when the mechanical actuators of the robot are set to the values corresponding to a particular pose. while Fig. thus we can write ˆ x W (ˆ.1 Generic Optimization Framework ˆ x Let the function W (ˆ. both the undeformed and deformed positions of our simulation mesh are allowed to depend on p. a condition we can easily detect by the direct solver not being able to factorize H. the physical system will assume the equilibrium state ˆ x xeq (p) = argmin W (ˆ. p) match a desired tarx get conﬁguration as closely as possible. this allows the skin to produce bulges and wrinkles where they did not appear before. 6 shows some shape variations that can be . To this end. We employ line search to make sure the energy decreases in each Newton step. A roughly area-preserving map is desirable for obtaining unbiased sample distributions. this information is not explicitly available from the 3D geometry such that we have to infer it ﬁrst. 6. All other values of x are either constant (constrained) or computable from p. ensuring with a sufﬁciently high penalty γ that the resulting x is close to a physiˆ cal solution with vanishing residual forces. xeq . including internal deformation energies and external contributions to the potential. X(pthk )) + Wext (ˆ) x x (13) The goal of the optimization will be to ﬁnd the optimal parameter values p such that the positions x(ˆeq (p).constraints. v) from a sparse set of height samples αi at positions (ui . x x (7) H ∆ˆ x = −f ∆p (12) for the increments ∆ˆ and ∆p. the motion of the actuators is physically restricted. First. we use a subset of points from high-resolution 3D scans (Sec. ˆ with the following objective function: E(ˆ. Fig. We cannot change the outside surface of the skin. we compute the second derivatives of the ﬁrst term in (10) only approximately. ignoring the third order derivatives of W that would result from applying the chain rule. It can happen that the matrix H becomes indeﬁnite. the actuator settings result in hard positional constraints that can directly be applied to the corresponding deformed positions. Practically. In the physical simulation. x contains a subset of the deformed ˆ positions x of the physical simulation. Although the skin clearly exhibits a thickness direction. Due to our choice of basis functions. the resulting deformation of the skin matches the pose’s target positions q as closely as possible. For ﬁxed parameter values p. we have to expose as many degrees of freedom as possible to the optimization process. We solve this as a minimization problem with respect to x and p. v). 4). This is measured in terms of a matching energy Ematch (x). but this is not a critical aspect in this context. We also cannot modify the attachment points of the actuators. p is a vector of generic parameters for which we want to optimize. In order to achieve the best results given these constraints. We construct a smooth warping ﬁeld w(u. we optimize the actuation parameters of the animatronic device for every target expression. p) ˆ x ˆ 6. We minimize (10) using a Newton method. v)+hn(u. We then compute the matching energy as Matching Energy. 5 illustrates the concepts of parametrization and MLS interpolation. where u and v are surface coordinates and the thickness parameter h runs along the surface’s normal direction n. For this we need to compute its ﬁrst and second derivatives with respect to x and p. Instead of working directly on the undeformed positions. Parameterizing the Undeformed State. + Ematch (x(ˆ. in which case we proceed as suggested in Nocedal and Wright [?] by adding a multiple of the identity to H.p ˆ The ﬁrst term penalizes the violation of condition (9). p). we deﬁne a warping heightﬁeld on the surface in order to smoothly deform space along its normal direction. While these computations are trivial ˆ for our particular choices of matching and regularization energies. we optimize with respect to two sets of parameters. the deformed position of a point can be computed as a linear combination of four nodal positions. vi )T using moving least squares (MLS) interpolation [?]. where f is the vector of ﬁrst derivax tives of E and H contains its second derivatives. We start by computing a low-pass ﬁltered version of the skin’s outer surface and construct a parameterization r : IR2 → IR3 using standard uv-mapping software. p)) + Ereg (p) (10) x x.2 Thickness Optimization (8) minimizing the total energy of the system. the squared distance between the deformed points and their desired target positions q.

va . Fabrication Constraints. We have ˆ ˆ To actuate the silicone skin we use a proprietary animatronic head developed by Walt Disney Imagineering. ui . thus the matrix ∂X/∂pthk is constant and can be precomputed. v) (1. Additionally. 6. Once the optimal αi are determined. v) amounts to ﬁtting an afﬁne transformation that minimizes the weighted least squares error θ(||(u. . In the objective function (10) we use ˆ x W (ˆ. vi ) ||)||a(u. Note that the MLS interpolation results in a linear mapping between undeformed positions X and parameters pthk . In this setting. . . there are several rigid attachment . vi ) − αi || .pthk ˆ i Ematch (x(ˆi . (19) The mapping from unconstrained nodal positions x and parameters pact to deformed positions x is ˆ xi = xi ˆ Ma (pact )Xi if node i unconstrained (20) if node i constrained by actuator a. this energy will tend to inﬁnity as any element’s volume approaches zero. Ereg (pthk ) = γundefreg W (X(pthk ). or falling below a user-deﬁned minimum thickness. where θ(x) is a weighting kernel and a(u. we use the regularization energy Ereg to keep the actuator parameter values within the range of physically feasible values. pact ) = W (x(ˆ. (a) Input geometry (b)–(d) examples of warped geometry. i m matching functions Ematch and the objective function can be written as m obtained for a simple one-dimensional example. Similar to the thickness optimization. evaluate the MLS warping ﬁeld to obtain wa = w(ua . abusing X(pthk ) as the deformed conﬁguration and the initial undeformed positions X(1) as the undeformed conﬁguration: Regularization. we found it most practical to reuse the deformation energy function W . x x Parameterization. Figure 6: Illustration of MLS interpolation using 7 sample points. va ) and ﬁnally compute the warped positions as Xa = r(ua . effectively preventing element inversions in the undeformed conﬁguration. (14) i T T T 2 E(ˆ. and use pthk = 1 for the MLS parameters. pact )). X(1)) (15) For the sake of simplicity. pact ). v) ∈ IR3 are the local coefﬁcients of the interpolation function. va ). Evaluating the MLS warping ﬁeld at an arbitrary parameter point (u. the given target positions qi should be matched as closely as possible. We need the undeformed mesh with nodal positions X(pthk ) to be a valid tetrahedral mesh where every element has positive volume. va ) + wa ha n(ua .3 Actuation Parameter Optimization The goal of the actuation parameter optimization is to ﬁnd the best values for controlling the mechanical actuators such that the resulting deformed skin matches a given target pose as closely as possible. The latter are determined by solving the normal equations for (14). we adaptively place the locations of the height samples in uv space based on a user-provided density map [?]. the ﬁnal rest conﬁguration is obtained in the following way: for each point Xa of the original mesh we ﬁrst retrieve its parameter values (ua . which can be done efﬁciently using precomputations. Regularization. the parameters pact of the optimization deﬁne the positional constraints to apply to the nodal positions x. Furthermore. 7 Animatronic Head We want to optimize the undeformed conﬁguration X(pthk ) such that m > 1 poses can be reproduced as closely as possible: for a set of actuator settings ai . Right: MLS warping ﬁeld shown as surface with isoparametric lines for illustration. X) + Wext (x(ˆ. v) − (ui . . To control the resolution of the warping heightﬁeld. we add a quadratic energy term to Ereg that penalizes MLS weights exceeding a value of one. This guarantees a fabricatable skin that does not intersect the underlying mechanical structure. The length of the red arrows illustrates the range of potential motion. Finally. xm and pthk at once. we gather the parameters αi of all sample points into pthk and compute optimal height values by minimizing (10). For any positive penalty value γundefreg . Multiple Pose Optimization.Figure 5: Left: parametrization of the geometry and placement of MLS samples. pthk )) x i=1 (17) (18) +Ereg (pthk ). We initialize the optimization with values xi computed from a perˆ pose forward simulation with incremental loading. ha ). pthk ) = x + i=1 m ˆ ∂W 1 γ 2 ∂ˆ x 2 (16) xi . While there are different ways to formulate and enforce this constraint. we linearize the transformation matrix Ma (pact ) of actuator a around the current parameter values pact in each step of the optimization. In this case our optimization is slightly modiﬁed and we optimize for all deformed positions x1 . It is driven by electric motors and features a = 13 parameters pact ∈ IRa to control the movement of the attachment links indicated in red in the ﬁgure on the left.

To obtain the speciﬁc animation parameters for a given facial expression. Silicone belongs to the class of elastically deformable polymers with a high yield strength. Our optimization process computes the volumetric shape of soft-tissue material under actuation such that the surface matches the deformation of given target surfaces. As matching criteria we select small surface regions close to the attachment links. We started by acquiring more than 100 frames of an expressive performance of our subject.35 mm. all experiments. Figure ﬁnishing included industry-standard techniques for adding hair and facial stubbles and painting a texture by a professional makeup artist. The difference between the scans of the real face and the simulation is larger. the target is not well approximated. To determine the operating range and control parameters of the head we attach a marker to each moving link and track its position under a number of sample locations and orientations. For our experiments.g. In our experiment. Tracking is performed using the same capture system as used for face scanning (Sec. the color can be adjusted. To better represent important facial deformation features on the forehead. it has a liquid consistency and can be injected into the mold. we use liquid injection molding to fabricate the physical skin as shown in the accompanying video. we used a subset of 8 expressive poses for thickness optimization. so in theory our optimization algorithm can be used with any animatronics device. and then showcasing our method with an example of real physical face cloning for an animatronic ﬁgure. This shows that our simulation is a good model for the real synthetic face. 9. As shown in Fig.links that constrain the movement of the skin. We validated our pipeline for measuring. 6. In those regions. 9 Results In this section we present our results.3. The mapping m(pact ) : IRa → IRl·6 from animation parameters pact to 3D locations and orientations of attachment points is then speciﬁed by quadratically interpolating the samples. we apply the same warping transformation to the captured geometry before doing comparisons (such as the error plots in Fig. modeling. we validate the simulation by comparing simulated deformations of our silicone skin with 3D scanned poses of the actual animatronic ﬁgure in Fig. The measurements are acquired by running an uniaxial stretch and compression test on a rectangular (55 × 5 × 2 mm — stretch) and cylindrical (50 mm diameter. 10) with our results. representing the limitations in the range of motion of the robot head and the range of deformation of the attached skin. We then aligned the neutral pose to a scan of the animatronic head. We showcase our complete pipeline by physically cloning a human face for an animatronic ﬁgure. 7. For all other regions we use our optimization process to achieve the desired deformation behavior by varying the skin thickness. After having optimized the ideal skin geometry. 9 shows our ﬁgure in action and compares it to the human face and the simulation under various facial expressions. 4). we additionally use a high-resolution mesh of ≈ 157k elements to optimize this area. We also scanned these poses of the animatronic ﬁgure and provide error visualizations showing the distance between the simulated surface and the scanned surface. our algorithm was able to exactly reproduce the spatially-varying thickness of the ground truth objects that were used to generate the inputs. In Validation Optimization. and compared the deformation of the predicted optimized shape to the real object. using material with Young’s modulus of E = 78kPa and Poisson’s ratio of ν = 0. we slightly warp the acquired 3D geometry of the human face to prevent intersections. with less than 4mm separating the surfaces everywhere. We start by generating ground truth data by randomly varying the thickness of a uniform block using the moving least squares warp as described in Sec. and the fabrication process is easy and safe. Fig. 4 shows a comparison of the force-strain curve of a simulated material to a measured sample. As can be seen from the error plots. Approximately seven days of room-temperature curing is required until it is ready to be used. The gap between the surface of the neutral pose and the underlying animatronics deﬁnes the initial unoptimized skin. Validation Simulation. We then used these target surfaces and the undeformed block as input into our optimization — see e. 12 mm height — compression) object. and simulating silicone materials.2. Our mold consists of a core that resembles the inner surface of the skin and six outer parts. our optimization process computes the thickness variation (c) such that the surface matches the deformation of the target under actuation (d). we also fabricated a subset of these examples. Because the animatronic head has very pronounced cheek bones. From the forward simulation of these objects we extract target surfaces. These are then used as hard boundary constraints in our forward simulation. starting with validation of our pipeline and optimization process. Fig. we solve the inverse problem as described in Sec. showing an average error of less than 1. Note that multiple outer parts are required so that the skin can be removed without destroying the mold. providing an intuition of which expressions are inside or at the border of the gamut of reproducible poses. Fig. For these 95mm long bars. we used the two-part silicone GI-245 by Silicones Inc. the agreement between the shape predicted by our simulation and the actual observed shape of the actuated skin is generally good. Consequently. Without optimization (e). We refer to the video for a visualization of the actuation level of each actuation parameter for a facial animation sequence. Fig.44mm (top) and 0. it is robust. After mixing it with the corresponding catalyst. The mold itself is fabricated using a rapid prototyping 3D printer.47 in its cured state. we also fabricated the optimized skin. 8 Fabrication Our choice to use silicone is based on the fact that the stiffness can be controlled accurately. We treat the underlying mechanics and controlling mechanism as a black box. Furthermore. which we represent as a tetrahedral mesh consisting of ≈ 63k elements. 6. the average errors between the deformed fabricated surface and the goal shape are 0. Comparing the human face to the cloned synthetic face.38mm (bottom). 8. Starting with a volumetric object (a) and a target surface (b). the large scale deformations and general expressions . Finally. We validated the optimization process in simulation. Face Cloning. 10 illustrates the unoptimized and optimized geometry and provides a side-by-side comparison of the deformation behavior. the surface thickness is constrained and cannot be changed because the skin is connected to the moving and static links. We then use our optimization process to obtain animation parameters for the entire captured face performance sequence. (a) (b) (c) (d) (e) Figure 7: Volumetric shape optimization.

4 (August). Cambridge Uni. B. B... P FISTER . AND BA JKA . N. D ONG . AND KOBAYASHI . R. M ATUSIK . M ATUSIK . Graph. C HENTANEZ . H. B ONET. AND F EDKIW. ACM Trans. T. Moreover. Limitations and Future Work. A.. K AUER . R. M. Press. e. R. Nonlinear Continuum Mechanics for Finite Element Analysis. W. M. 29 (July). Graph. AND G UO . H. 2005. B ICKEL . L.. (Proc. we currently only use a single soft tissue material to produce a synthetic skin. Finally. M. C HO . M.. W. 2010. This work was supported in part by the NCCR Co-Me of the Swiss NSF.. Wavelet importance sampling: efﬁciently evaluating products of complex functions. 1166–1175. H ARA . ACM Trans. Relaxing this constraint would likely give us more degrees of freedom... G OLDBERG . In Proc. M. D. In Comp. P. A KAZAWA . We believe that our computationally guided process for designing and fabricating animatronic characters will serve as a blueprint for how future robots with soft tissue should be built. G ROSS . 2007. Because this work is the ﬁrst in the area. H EIDRICH . Graph.are captured well. 131–140. 2010. G OURRET. J. ACM Trans.). 29 (July). B.. 2009. P OPA . R. M. S UMNER . 3 (July). 29. P FISTER .. B. H. W. B EARDS LEY. M.. B. JAROSZ . AND B UTTAZZO . AND T HALMANN . B EELER . Acknowledgements We would like to thank the anonymous reviewers for their comments. A. J.. CVPR.. Graph.. WANG . AND G ROSS . 10 Conclusion In this paper we have described a process for computationally guided design of an animatronic character composed of a soft-tissue material and an electromechanical base. ¨ B ICKEL .. H AUSER . ACM Trans. A. ACM Trans. P FISTER . P.. 3 (Aug. F. M. V.. 3. Our experimental validation shows that using this process provides a principled way to design soft-tissue animatronic characters. D. D. ACM Trans.. T ERAN . 26. T ONG . Design and fabrication of materials with desired deformation behavior. A LTEROVITZ . D. 257–287. and the differences show where improvements should be made in the physical system to enable the ﬁtting methods to achieve a better match. ¨ B ICKEL ... High resolution passive facial performance capture. K. Accurate multi-view reconstruction using robust binocular stereo and surface meshing. High-quality single-shot capture of facial geometry. B ACHER . In Proc. ACM Trans. H AHN . 1989. References B ECKER . I RVING . and hair. 4 (July).. 61:1–61:10. B ACHER . M. P. Medical Image Analysis 6. B ICKEL ... W. in our current framework the attachment of the actuators is ﬁxed to speciﬁc places underneath the character’s skin. B UCUR . D. We believe we can extend the range of possible deformations by using multi-layered materials similarly to Bickel et al. 2010.. 24. K. 3 (July). M. A KENINE -M OLLER . Capture and modeling of non-linear heterogeneous soft tissue. H. K. 89:1–89:9. B OTSCH .. H. 29. B ICKEL ... of IEEE International Workshop on Robot and Human Interactive Communication. 2010. . B OUBEKEUR .. OTADUY.. it is important to note that the motions of current animatronic characters are still limited as they are less expressive than most humans. AND T ESCHNER . Realistic facial expressions by sma driven face robot. Invertible ﬁnite elements for robust simulation of large deformation. In SimVis. 41:1–41:10. Multi-scale capture of facial geometry and motion. L EE . A NGST. Simulation of object and human skin deformations in a grasping task. V USKOVIC .. 29. Graph... G. B RADLEY. First. In 2004 ACM SIGGRAPH / Eurographics SCA. 33:1–33:10. 2009. S UMNER . Graph. 4 (July). 28. 4 (July). 2011. B RADLEY. B EARDSLEY.. J.. 2008. OTADUY. T HALMANN . AND M ATUSIK .. M ATUSIK . ACM Trans... Graph. N. [?]. AND RUSINKIEWICZ . it is important to develop retargeting algorithms that can map the expressions of a target human to the physical constraints of the electromechanical base and materials used. J. AND W OOD .. and Lee Romaire for ﬁgure ﬁnishing. 63:1–63:10. 2002. Robust and efﬁcient estimation of elasticity parameters using the linear ﬁnite element method. 504 –511. In this paper.. W. B. 2004. T. T. 1997. Variational Methods in Shape Optimization Problems.. ACM Trans.-P. SIGGRAPH). B. M. Graph. X. D... AND O’B RIEN . M. H. Physical reproduction of materials with speciﬁed subsurface scattering. G. while in this work we have only shown examples of replicating human faces. D UAL . H. Inverse ﬁnite element characterization of soft tissues. H. High-quality passive facial performance capture using anchor frames. we predict that similar frameworks will be used to design realistic full-body characters..... Interactive simulation of surgical needle insertion and steering. F.... R. ACM Trans. OTADUY. D.. J.. Brian Tye. it has a number of limitations that should be addressed in future research. T. the rest shape of the soft tissue would not need to be the same as the rest shape of the electromechanical base. AND H EIDRICH .. F UCHS . M. 28.. and Walt Disney Imagineering for their support. 88:1– 88:10.. M. AND G ROSS ... AND G ROSS . B EELER . F. ˇ H A S AN . We compute a volumetric shape of the soft-tissue material and a set of actuation forces that can reproduce target deformations of an exterior character surface. Y.. 15–28. W. 21–30. M. subsurface scattering. W. R ITCHIE . Graph. The core requirement of this process is a physically based simulation framework that lets us accurately predict the behavior of the soft tissue when subject to external forces. Graph. G OTSMAN . S. W.. ¨ C LARBERG .. S ZEKELY. we have not addressed the appearance aspect of the synthetic skin tissue including reﬂectance. Frank Mezzatesta. F. 2001... R. B RADLEY. 62:1–62:10. As an example. J. Birkhuser Mathematics. 75:1–75:10. 30. M. Graph. S HEWCHUK . 2007.. J. C. AND G ROSS . 3 (July). 2005.g. AND J ENSEN . M. T. P ELLACINI . G. this process allows us to construct an animatronic head that approximates the shape and behavior of a real person whose expressions are acquired using a 3D scanning system.. Second. This is also a very interesting direction for future work and we envision that similar computationally guided processes could be developed to solve these problems. Fabricating spatially-varying subsurface scattering. 2010.. Thus. AND S HEFFER . M. R. W. 40:1–40:9. For further comparisons we refer to the accompanied video.. P FIS TER .

error comparison of our simulation to the fabricated skin (blue 0mm. Figure 9: Side-by-side comparison. red 4mm). Renderings of 3D reconstructions. expressions generated with the optimized fabricated skin actuated by an animatronic device. Columns (d) and (e) show the undeformed and deformed state of the corresponding fabricated objects against a 4mm grid. error comparison of 3D reconstructions to the fabricated skin (blue 0mm. .(a) (b) (c) (d) (e) Figure 8: Illustrating the volumetric shape optimization in 1D. our optimization process computes the volumetric shape (b) such that the surface matches the deformation of the target under actuation (c). red 4mm). From left to right: Photographs of captured facial expressions. Given a volumetric object and a target surface (indicated in orange) as shown in column (a). 3D simulation of the animatronic head for the corresponding pose.

T ERZOPOULOS . 2009. Springer. T HOUTIREDDY. . L. Numerical coarsening of inhomogeneous elastic materials.. P.. ACM Trans. D.. IEEE Trans.. VAN G ELDER . H. Conference on Intelligent Robots and Systems.. T ERZOPOULOS . S. J E R ABKOV A . K. C. K. Graph. 569– 579. 1998.. ¨ KOCH . L I . SIGGRAPH ’87. SIGGRAPH) 27. J.. H ANSON . W. L EVIN . red 4mm). K HAREVYCH . N ISHIO .-M. L. K. D.. High resolution acquisition. of Comp. Design of android type humanoid robot albert hubo. and int.. 1996. D. 677–686. H. R ICHMOND . D. E. 2004. S AMA RAS .-S. Simulating facial surgery using ﬁnite element models. 99:1–99:17. AND H UANG . K. Graph.. N G -T HOW-H ING . The approximation power of moving leastsquares. WANG . L AU . S. VON B UREN . M ULLEN . Z HANG . 67 (October).-H. Preserving topology and elasticity for embedded deformable models. L EE . E. S. M ATUSIK . N. H. no. In ACM SIGGRAPH 2006 Papers... R. L EE .. 2006. 2007.. K RY. AND G ARTNER . B LEMKER . J. 3. In Proc. tech. In ACM SIGGRAPH 2009 papers. D. 2005. M. Trans. Pattern Anal. Automatic determination of facial muscle activations from sparse motion capture marker data. 11.. H.. 51:1–51:8. AND T ERZOPOULOS . R. M ORI .-H. OWHADI .. L LOYD . O H . ACM Trans.. (b. Tools 3. 1517–1531. 29 (July). J U . Z. JAMES .. Musculotendon simulation for hand animation. K IM . D. 15 (June).. J. ACM. Intell. O. Comprehensive biomechanical modeling and simulation of the upper body. F. of IEEE/RSJ Int.... I SHIGURO . Numer. Fabricating microgeometry for custom surface reﬂectance. T. C..-Y. Meth. BARR . graph. 2009. I. E. ˇ´ ´ N ESME . 2005.. Y. ACM. J. M ETAXAS . T ERAN . L. AND FAURE . In ACM SIGGRAPH 2007 papers. 52:1–52:9. X. S.. In Proc. 205–214... Graph. AND O RTIZ . D . P. 158–179... 87–96. AND I GARASHI . and int. 3. AND F EDKIW. M. 2000. Elastically deformable models. 21– 42. L EE . Comparison of the unoptimized and optimized skin: Error plot of the (e) unoptimized and (f) optimized skin (blue 0mm. 1987. C. D.. S HEN . I. M..-S. P EERS . P.-H. ACM... C ARLS . (b) Target geometry. Graph.. Analysis and synthesis of facial image sequences using physical and anatomical models.. J.. 61. Graph. J. Comp..).. 2001. Geminoid: Teleoperated Android of an Existing Person. I. S... D. A variational r-adaption and shape-optimization method for ﬁnite-deformation elasticity.-Y. AND PAI . S IFAKIS . 3 (Aug. L I .. AND T ERZOPOULOS . graph. SIGGRAPH ’07. 2004. L. S IFAKIS . H UANG . S UEDA . Graph. and int. Plushie: an interactive design system for plush toys. graph. R. K AUFMAN . on Vis. S. Comput. L ANG . AND F EDKIW. 2009.. ACM Trans. and Comp. A.. Creating and simulating skeletal muscle from the visible human data set. R. 2006. S. Engng.. 417–425. V. In Comp.. S. W EYRICH . S NAVELY. I. Spacetime faces: high resolution capture for modeling and animation. 2010. 28 (September).. Heads up!: biomechanical modeling and neuromuscular control of the neck. August. 1993. ¨ S CHENK . AND H U . S.. M. W. (Proc. C URLESS . 2. a comparison of the fabricated unoptimized (g) and optimized (h) skin shows visible differences. AND H AGITA . I-Tech. Mach. D. In Elec. AND D ESBRUN .. X. Scanning physical interaction behavior of 3d objects.. IEEE Trans. S. A. G ROSS . AND YAU . 317–328. D. H.-S. 1188–1198. 1–21.. ACM Transactions on Graphics 23. S. FANKHAUSER . ACM Trans. T. K. A. learning and transfer of dyanmic 3-d facial expressions. Forum. ACM Trans..c) Rendering of the inner side of the unoptimized and optimized forehead with color-coded thickness variation.. tech. 28 (July).. AND S EITZ .. Math.. H AN .-H. 111:1–111:9. AND WATERS . Numerical Optimization. 32:1–32:6. 28 (July). 2004. M. 1998.(a) (c) (e) (g) (b) (d) (f) (h) Figure 10: Forehead Optimization: (a) Rest shape.. Int. On fast factorization pivoting methods for symmetric indeﬁnite systems. Approximate simulation of elastic membranes by triangulated spring meshes. ACM. N. Y. D. T. G.. D. N OCEDAL .-W. AND RUSINKIEWICZ . F. Z HANG . 1428–1433. ACM. Graph.. Although the variations are subtle.. 421–428. 2009. P. In ACM SIGGRAPH 2005. K IM . S. AND PARISH . V. In Proc. N EVEROV. of Comp. SIGGRAPH ’96.. E LGAMMAL .. S. In Proc.. J. Numer. P LATT. AND F LEISCHER . J. Y. tech. K. Y. J. L.. ACM. 23. 2008. Humanoid Robots: New Developments.. E. AND W RIGHT. S IFAKIS . H UANG . 2006. Popup: automatic paper architectures from 3d models. 548–558. B. PAI . 2007. A. Graph. M. G.. ch. J. SIGGRAPH ’01. J. D OEL . AND PARK . P. F.

- ASEE-2012 Integrat Business Process Simulation Softwareuploaded bymasterwolfx
- 유럽 SAMCO issue 09.pdfuploaded bystormstone
- Assignment 3 : Simulation and Stella Reportuploaded bySHAFIQAH HANAFI
- 062uploaded bysiditude
- Games in Career Guidance Effectiveness of Using auploaded byJudith Fetalver
- Surech 2014uploaded byFABIAN FIENGO
- 455984.fucak_fpuploaded byZoric Bobby
- Paceuploaded byAvdhesh Sharma
- Untitleduploaded byapi-125299886
- Base Paperuploaded bySwapna Kumari
- 8-4-14 Prog 2uploaded bywararm
- Assignment 3 : Simulation and Stella Reportuploaded bySHAFIQAH HANAFI
- Validation of multi train simulation software.pdfuploaded by曾乙申
- Dada Sampleuploaded byPauloSilver
- Turbine Blade1uploaded bysirajul_musthafa
- 3-Energy Analysis Using Building Elementsuploaded byaomareltayeb
- Crawley - Contrasting the Capabilities of Building Energy Performance Simulation Programsuploaded byHamo Hamo
- Using Behavior of Transition Translation Traceabilityuploaded bysfofoby
- Relevance of Motion-Related Assessment Metrics in Laparoscopic Surgeryuploaded byRobert Peterson
- 410_test_2_part_1_sp12uploaded byimran_chaudhry
- simulation scenario templateuploaded byapi-258761055
- Pipesim.pdfuploaded byClemen
- Taiba - IES VE Training 2 of 2uploaded bySyed Moazzam Ali Ali
- 10.1.1.1.5365uploaded byaeroacademic
- MELJUN CORTES CSCI19 SYLLABUSuploaded byMELJUN CORTES, MBA,MPA
- Fan Blade Bird Strike Analysis Using Lagrangian,uploaded byrs0004
- scimakelatex.27207.iPhone+7uploaded byLK
- Manual Simulator TechSimuploaded byAlefterix Anca
- Assessing Delays_ the Impacted as-planned Method Compared to the Time-impact Analysis Methoduploaded bySenathirajah Satgunarajah
- 27_ vol_7no_2_2012.pdfuploaded byGokul Prabu

- SRK v NIST erroruploaded byTaylor W. Hickem
- Rotating Pump Traininguploaded byTaylor W. Hickem
- Material Compatibilityuploaded byTaylor W. Hickem
- Wage premium example worksheetuploaded byTaylor W. Hickem
- USDOT - Pedestrian and Walkway Design Guidelinesuploaded byTaylor W. Hickem
- Combustion Calculationsuploaded byTaylor W. Hickem
- Turbine helpful introuploaded byTaylor W. Hickem
- Pilot data sheet blankuploaded byTaylor W. Hickem
- Safe Location of Oxygen and Inert Gas Ventsuploaded byTaylor W. Hickem
- An Equilibrium Characterization of the Term Structureuploaded byTaylor W. Hickem
- FLUKE TI32 USERS MANUALuploaded byTaylor W. Hickem
- Zhang - Stock Market Forecasting using Machine Learning Algorithmsuploaded byTaylor W. Hickem
- Employment Act Chapter 91 Part III - 23 - payment on termination by employeeuploaded byTaylor W. Hickem
- Fudihara - A Numerical Investigation of the Aerodynamics of a Furnace With a Moveable Block Burner - 2006uploaded byTaylor W. Hickem
- Recent Advances in Creep Resistant Steels for Power Plant Applicationsuploaded byTaylor W. Hickem
- Survey Tactile Human Robot Interactionsuploaded byTaylor W. Hickem
- Distributed Power Adjustment in Cognitive Radio Networks Using PID Controluploaded byTaylor W. Hickem
- PID Tuning Rules - Skogestad 2001uploaded byTaylor W. Hickem
- Buchanan James M - Cost and Choice 1969uploaded byTaylor W. Hickem
- No Flow Underfillsuploaded byTaylor W. Hickem
- End of Lifeuploaded byTaylor W. Hickem
- WEF GlobalRisks Report 2012uploaded byTaylor W. Hickem
- Warning on ballast failuresuploaded byTaylor W. Hickem
- End of Lifeuploaded byTaylor W. Hickem
- Lessons learned in furnace failuresuploaded byTaylor W. Hickem
- Women's Charteruploaded byTaylor W. Hickem
- Theory and Application Reciprocating Compressors PDHEngineeruploaded byTaylor W. Hickem

Close Dialog## Are you sure?

This action might not be possible to undo. Are you sure you want to continue?

Loading