CGW.COM | EDITION 2 – 2020 | $8.00 US | $8.

25 CAN

PLUS
PIXAR SPARKSHORTS
MEDICAL VIS
LIGHT FIELD PRIMER
MYTH: A VR STORY

MAKING
MAGIC
CRAFTING THE LEGENDARY
VFX IN CURSED

1 cgw s u m m e r 2019

CGW_Issue2-2020.indb 1 8/7/20 8:12 AM

AD-2019_Q2_SMC_WorkStation_Banner.indd 1 4/28/2020 10:23:31 AM

CGW_Issue2-2020.indb 2 8/7/20 8:12 AM

 CGW
features
16 Igniting Passion
Pixar’s SparkShorts program
gives employees the chance to
bring their own animated short
films to life.
COMPUTER GRAPHICS WORLD
Universal Scene
22 Description
Pixar does a deep dive
into USD and its effect on

on the cover content creation.

30 A Bedtime Story

8
Making of a Legend
Netflix turns the Arthurian
myth on its head with
a unique story that is
augmented with enchanting
visual effects.
Filmmakers use VR to extend
the world of the elemental spirits
in “Myth: A Frozen Tale.”
34 Left Brain, Right Brain
Animators use computer
graphics to help explain complex
medical concepts.

Visit CGW.com for Web-only features and news.

38 Working Together
For M&E workstation
performance, it’s different
Plus, stay abreast of the latest industry changes and strokes for different folks.

updates resulting from the COVID-19 epidemic.

40 Transcending
Boundaries
41 Dark Days
45 Pre-production

departments in Real Time

50 Spinning a Yarn

2
Netflix’s The Willoughbys –
Editor’s Note Not your typical tale.
SIGGRAPH “Thinks Beyond” the norm and revamps its
annual conference and exhibition in this very unusual

56
time – all the while incorporating nearly all the familiar
elements into the virtual rendition.
Education
• Classroom Disrupted

4
• Ahead of the Curve
Spotlight

See it in Post • J.J. Sedelmaier Productions: 30 years

60 Hybrid Drama
The Blacklist’s out-of-the-box
production strategy
JULY/AUG 2020 POST
www.postmagazine.com
creating commercials & pop-culture classics during the pandemic.
PLUS:

• mixing sci-fi & satire for Amazon’s Upload 

DAVID FINCHER’S MINDHUNTER
UPLOAD DIRECTOR
GREG DANIELS
PIXAR’S SPARKSHORTS
W H E R E T E C H N O L O G Y A N D TA L E N T M E E T S M
J.J. SEDELMAIER TURNS 30

• stay in tune with the latest in Music Videos

• Director David Fincher on 63 Quest for Reality
A primer on light field

MAKING Netflix’s  Mindhunter technology.

MUSIC VIDEOS
MUSHROOMHEAD, MACHINE GUN KELLY,
DIPLO & MORE!

POSTMAGAZINE.COM

e d i t i o n 2, 2020 c gw 1

20 10:23:31 AM

CGW_Issue2-2020.indb 1 8/7/20 8:12 AM

 ADAPTING TO THE NEW NORMAL CGW COMPUTER GRAPHICS WORLD

B
ack in March, when NAB was canceled abruptly due to COVID-19, it was difficult
to envision the new normal beyond the short term. Perhaps it was naivete or THE MAGAZINE FOR DIGITAL CONTENT PROFESSIONALS

ignorance on my part, and while it became obvious soon enough, at the start of
the outbreak i never imagined that SIGGRAPH might suffer that same fate.
However, after 46 years of adapting and serving the needs of the computer graphics
EDITORIAL
EDITOR-IN-CHIEF Karen Moltenbrey
industry, SIGGRAPH has proven how truly agile it can be, as the show management and
e: karen@cgw.com t: 603.432.7568
volunteers for this amazing conference and exhibition have transitioned it to a virtual plat-
DIRECTOR OF WEB CONTENT Marc Loftus
form due to the ongoing pandemic. While some other trade shows have gone virtual with e: mloftus@postmagazine.com t: 516.376.1087
an extremely scaled-down offering, not so with SIGGRAPH. Instead, it is offering its exten- CONTRIBUTING EDITORS Kathleen Maher, Jon
Peddie, George Maestri, Barbara Robertson
sive complement of events and programs, from a virtual exhibition area, to its Computer
PUBLISHER / PRESIDENT / CEO William R. Rittwage
Animation Festival, to its Technical Papers, Panels, Courses, Emerging Technologies, Job
COP Communications
Fair, Immersive Pavilion, Keynote, Production Sessions, Real-Time Live!, Labs, VR Theater,
and more. The various chairs and committees have devised the most optimal methods ADVERTISING SALES
of bringing each element of SIGGRAPH to this year’s virtual attendees. It seems that this DIRECTOR OF SALES—NATIONAL Mari Kohn
year’s theme of “Think Beyond” became even more relevant than anyone initially thought. e: mkohn@copcomm.com t: 818.291.1153 c: 818.472.1491

Speaking of SIGGRAPH, the Computer Animation Festival’s Best in Show award goes CORPORATE SALES EXECUTIVE—
EVENTS, CUSTOM AND INTEGRATED PRINT/
to Erica Milsom for “Loop,” part of Pixar Animation Studios’ SparkShorts program. Read PUBLISHING SERVICES Lisa Neely
more about this program and the making of “Loop” in this issue. Meanwhile, we are also e: lneely@copcomm.com t: 818.660-5828

presenting to our readers a behind-the-scenes look at the making of the experimental EDITORIAL OFFICE / LA SALES OFFICE
620 West Elk Ave., Glendale, CA 91204 t: 800.280.6446
Pixar VR short film “Myth: A Frozen Tale.”
Typically in this issue, CGW covers the year’s summer VFX and animated theatrical
ART/PRODUCTION
blockbusters. This year, however, with film production at a near standstill, we have looked
SENIOR ART DIRECTOR Michelle Villas
beyond the big screen and have found no shortage of amazing series made with amazing e: michelle@moontidemedia.com
technology for smaller screens, including the newly released Netflix’s Cursed.
ONLINE AND NEW MEDIA Elvis Isagholi
While many studios were establishing remote workflows due to the virus, there are e: eisagholi@copcomm.com
some animators whose job it is to work intimately with the novel coronavirus and similar
concepts in the field of medical visualization. We introduce you to some of these unique SUBSCRIPTIONS
experts whose work straddles the line between science and art. 818.291.1117
We round out the print issue by examining an important technology: light fields.
But wait, there’s more. We have had a long lead time since our last issue in early April, CUSTOMER SERVICE
made even lengthier since SIGGRAPH moved back a month. As such, we have been e: jlopez@copcomm.com t: 818.291.1117

accumulating stories, stories you will no doubt still enjoy. Therefore, we are publishing an
expanded digital version of this issue, with more great content. You can get it by signing
up for this and other digital issues on CGW.com; you also will be able to find the stories on
our website when we make the current print issue available there.
Our bonus content includes coverage of the animated feature The Willoughbys as well
as the live-action spy drama The Blacklist, which finished out its season with a hybrid
live-action/animated episode when lockdown occurred.
With Emmy Awards on the horizon, we also take a deep dive into two other captivating
series: Amazon’s Carnival Row and HBO’s His Dark Materials, as the film hiatus provided us
with additional time (and space) to cover some well-deserving projects that did not make
it into our last issue. We also take a close look at Universal Scene Description (USD).
As a departing note, I want to applaud all the resourceful technologists in our industry
who, with a moment’s notice, were able to get their studios and remote teams up and
running so they could continue working on projects and keep the industry moving forward
during this tough time. To everyone else, stay positive in this time of our new normal. 
COMPUTER GRAPHICS WORLD MAGAZINE IS
PUBLISHED BY COMPUTER GRAPHICS WORLD,
A COP COMMUNICATIONS COMPANY.
Computer Graphics World does not verify any claims or other
information appearing in any of the advertisements contained in
the publication, and cannot take any responsibility for any losses or
other damages incurred by readers in reliance on such content.
Computer Graphics World cannot be held responsible for the safe-
keeping or return of unsolicited articles, manuscripts, photographs,
illustrations or other materials. Address all subscription correspon-
dence to: Computer Graphics World, 620 West Elk Ave, Glendale, CA
91204. Subscriptions are available free to qualified individuals within
the United States. Non-qualified subscription rates:
USA—$68 for 1 year, $98 for 2 years;
Canadian subscriptions —$98 for 1 year and $136 for 2 years;
all other countries—$150 for 1 year and $208 for 2 years.
Digital subscriptions are available for $27 per year.
Subscribers can also contact customer service by calling
818-291-1158, or sending an email to csr@cgw.com.

Postmaster: Send Address Changes to

Computer Graphics World, 620 W. Elk Ave., Glendale, CA 91204
Please send customer service inquiries to
620 W. Elk Ave., Glendale, CA 91204

Karen Moltenbrey,
Editor-in-Chief RECENT AWARDS
karen@CGW.com

2 cgw e d i t i o n 2, 2020
 FYC.NETFLIX.COM

COMPUTER GRAPHICS WORLD (CGW)

CGW_Issue2-2020.indb 3 8/7/20 8:12 AM
 SPOTLIGHT
Virtual studio in the cloud, simplified
Lucky Post, a full-service post-production studio in Dallas and Aus-
tin, Texas, that combines creative editorial, color, VFX, finish, motion
graphics, and sound design, found itself in the unenviable position of
many post facilities amid the COVID-19 crisis. It had to find a way to
keep its employees working remotely. That’s when media systems
integrators CineSys-Oceana introduced them to SimpleCloud, a
cloud-based platform for digital content focused on animation,
architecture, editorial, game development, and visual effects. Sim-
pleCloud provides management and setup of a virtual studio in the
cloud, allowing users to be connected to a virtual workstation from
anywhere, through any thin client or HTML5-supported browser.
Here, Tim Nagle, senior Flame artist at Lucky Post, discusses
his team’s experience integrating and using SimpleCloud for
remote work.
What initial challenges did you face?
The most obvious is how to give our clients a seamless experience
and give the artists the power to collaborate like they did every day
prior to this. How are we meant to start or finish a job, and focus on
creativity, when we are removed from our infrastructure?
Some of the top requirements we had were collaboration for
artists, speed and efficiency with the tools, and making our clients
successful while we are all in 16 remote locations.
Did SimpleCloud fit those needs?
We found that SimpleCloud, combined with our Evercast stream-
ing solution, was the most efficient way to move forward with
active projects and spin up new projects in a virtual environment.
We arrived at this solution with Jason Starne from CineSys-Ocea-
na. We have had a long relationship with CineSys, and we know
and trust Jason to jump in and help us find a solution whenever we
need one.
We did not try other products prior to SimpleCloud. When we saw
4 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 4
its power, scalability, and compatibility with our workflow, it was the
clear solution.
What does your workflow look like?
We have four editors, five assistant editors, one graphics designer,
two sound designers, one colorist, and two Autodesk Flame artists.
With SimpleCloud, we’re able to mirror our typical workflow in a
remote environment. We utilize shared storage to carry a job from
off-line through finish, similar to how we would at our physical studio.
And the results?
This has been an amazing experience. We were able to add some
customizations to workstations to leverage Evercast and rely purely
on the data center network connection, while the clients logged on
to supervise. In these times, you can imagine the stress on the local
network nodes with everyone working from home; this solution has
mitigated any issues with local upload speeds from artists’ homes.
What applications do you use in SimpleCloud?
We are using the Adobe Suite, Maxon’s Cinema 4D, along with some
customizations we asked for to solve our streaming needs.
How has it worked so far?
This has been amazing for us to be able to serve our clients seam-
lessly, as if we physically have the infrastructure of our facility.
How was your experience with CineSys?
CineSys-Oceana has been integral to our success in this transition.
We rely on them to help solve problems creatively and sustainably.
Do you have any advice for other companies that want to
work remotely?
We suggest anyone trying to navigate the challenges of working
remotely while giving your clients the same experience, consider
SimpleCloud and don’t be afraid to customize where needed.
8/7/20 8:12 AM
 A note from the CGW team to our readers:

We are aware that the COVID-19 crisis has forced us to

adjust in different ways. In light of studio and office closures,
we’d like you to know you can redirect your subscription to a
residential address – quickly and at no charge. Simply visit
http://www.cgw.com/Subscriptions.aspx to update your
mailing preference. You can also receive our digital edition at
no charge.

We are committed to continuously delivering timely editorial

coverage via all of our platforms today, just as we have done
for the past four decades.

We are in this together. Stay strong, stay healthy,

and stay connected.

facebook.com/CGWmagazine @CGWmagazine

CG
W.C
O M|
ED
ITIO
N1
IA L –2
02
P EC 0|
YS $8
.00
AR
RS
US
E |$
NIV
8.2
5C
AN AN

HE DIGITA
LLY DE
PLUS
M T G! -AGING
RO
...F INNIN UNDE
ACTO
RS
B EG RCOV

W
SPIES ER WIT
IN DISG H
N KLAUS: UISE
CA A TRAD
.25 ITIONA
| $8 ANIM LLY

G
US ATED
.00 WHY W TREAT
| $8
2019 ORKS
R TATION
M ME CONT S SEE
SU INUED

C
O M| GROW
W.C TH
CG

OR LD
SW
PHIC
GRA
P UTER
COM

ICEBREAKE
NORD
COME
IC MYT
ALIVE
HS AN
D
IN FROZ LEGENDS
EN 2
R

I E N D S, ELFIN
FR NDEED MAG ’
I PIX
AR
IC ’S
LA
CGW_Covid.indd 1
CGW_2020-Issue1.indb 9 TE
ST
QU
4/6/20 11:58
4/9/20 11:44 AM
E
 SPOTLIGHT
Global PC Gaming Hardware Market Forecast
Expects Surge in 2020 Due to COVID
Jon Peddie Research (JPR), the industry’s research and consulting
firm for graphics and multimedia, reported in its newest gaming
market study regarding the PC Gaming Hardware Market, which
consists of personal computers, upgrades, and peripherals used for
gaming. The market is booming globally due to shelter-in-place or-
ders as gamers upgrade and buy new PCs and accessories. Perhaps
more importantly, the current situation has actually created millions
of new PC gamers looking for immersive, exciting, and economically
efficient forms of home entertainment.  we still predict growth over five years even in the face of a console
Ted Pollak, senior analyst–gaming industry, says, “The PC Gaming
Hardware Market is in a rare scenario where every segment is going
up. We see a lot of people buying and upgrading personal and com-
pany subsidized computers with better parts, with the intention of
playing video games. In the Entry-Level, much of this revenue comes
from new gamers.”   The “Worldwide PC Gaming Hardware Market” report series
The 2020 Entry-Level category is forecast to grow 21.7%, which
is unprecedented and totally unexpected. The Mid-Range has
bounced back from a slide; now in positive territory. At the High-End,
1440p+ display sales (spurred by more affordable offerings) created
a chain reaction of upgrades as gamers configure rigs for 60+
frames per second.   per issuance.
Collaborate and work from
anywhere
The secure, global, cloud-based
platform for digital content
management and creation,
perfect for Animation, VFX,
Advertising and Games
Powered by:
6 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 6
Due to TV broadcasting of sim racing events, we are observing an
uptick in racing simulation builds. These include a high performance
computer often with premium audio, racing wheel/shifter/pedals,
sometimes a racing seat, and other elements. Many of the sim
racers are new to the market but have money and spend $2,000 to
$5,000 on systems, audio, and accessories.” 
Jon Peddie, president of JPR, says, “COVID-19 has created a big
spike in sales for PC gaming products in the near term. The total
market is approaching $40 billion in 2020. However, we temper our
forecasts for possible economic issues in 2021 and beyond. Fore-
casts are heavily dependent on consumer confidence. Nevertheless,
T
cycle. We have improved our accessory analysis this year and added
a new category for game recording and streaming products. Studio-­
quality cameras and microphones, capture cards, and personal
lighting have become common items for high-end gamers and are
making their way down into the mass market.” 
by Jon Peddie Research – which covers 33 countries, notebooks,
desktops, DIY, and accessories – comes in three versions: Entry-­
Level, Mid-Range, and High-End, with a subscription price of
$27,500, including a global summary. Subscribers receive all re-
ports twice a year. Reports are also available by segment at $8,250
8/7/20 8:12 AM
 Collaborate and work from anywhere

The secure, global, cloud-based platform for

digital content management and creation,
perfect for Animation, VFX,
Advertising and Games

CGW_Issue2-2020.indb 7 8/7/20 8:12 AM

 Making of
a Legend
NETFLIX’S ‘CURSED’ TURNS THE ARTHURIAN MYTH ON ITS HEAD

BY KAREN MOLTENBREY

Images ©2020 Netflix

8 cgw e d i t i o n 2, 2020

CGW_Issue2-2020.indb 8 8/7/20 8:12 AM

 T
here have been many retellings and in- Wheeler and Miller are co-creators/execu- puter-generated creatures and animals, and
terpretations of the Arthurian legends, tive producers, while Silenn Thomas serves fire/ash/embers/smoke/water. Most of the
particularly those involving Excalibur. as co-executive producer, and Alex Boden, Fey, however, use prosthetics.
And with the magic, drama, romance, myth- a producer. David Houghton is overall visual “One of the decisions we made early on
ic battles, epic quests, and more, it’s easy to effects supervisor. in conversations with Frank and Tom was
see why the subject is so spellbinding. And Cursed is set in a mythical medieval version that the magic was going to be visualized
indeed, the latest version of this medieval of England, although filming occurred at through elemental forces rather than more
myth, Netflix’s Cursed, follows that tradition, Langley Film Studios, outside of London, with traditional wand/firework-type effects,” says
although it places a unique spin on the tale a good deal of location shooting near Surrey Houghton, “more embedded in elemental
with familiar characters in very different and in North Wales, around an area called forces.” This can be seen in many scenes,
roles, all supported by a wide range of stun- Betws-y-Coed. VFX were used to transform such as when Nimue summons the Hidden
ning visual effects. the soundstage and modern locations. with animated vines representing the
Based on the illustrated novel by Tom All told, the series contains 2,051 VFX mystical, natural forces that give the young
Wheeler and Frank Miller of the same shots: Episode 1 (390 shots) and Episode woman some of her powers; or when she
name, Cursed re-imagines the story from 10 (360 shots) contain the highest number, manipulates smoke, flames, and ash to her
the perspective of Nimue, a young woman followed by Episode 4 (282 shots), Episode advantage. Lightning also becomes a natural
with a mysterious gift who is destined to 2 (261 shots), and Episode 7 (229 shots), force that is manipulated by magical char-
become the powerful (and tragic) Lady of with the remaining dispersed among the acters in the show.
the Lake. After her mother’s death, she finds other episodes. The work was shared mainly
an unexpected partner in Arthur, a humble across five vendors: DNeg, Milk VFX, Mr. X, DNeg: Hawksbridge, Gramaire,
mercenary, in a quest to find Merlin and Goodbye Kansas, and Freefolk. Village Massacre, Vines, Bear
deliver an ancient sword. Over the course of “We did just about everything,” says DNeg’s work across its 822 shots in Cursed
her journey, Nimue will become a symbol of Houghton. “I’m not sure what we didn’t do; spanned the series’ 10 episodes and ran the
courage and rebellion against the terrifying we didn’t do spaceships.” There is quite a gamut, incorporating just about every type
Red Paladins – Christian Crusaders whose bit of blood from battles, massacres, and of work, from environments, to creatures
goal is to rid the land of druids and all mys- brutality, as well as magical effects, all-CG and digi-doubles, to fire and blood, and far
tical/magical beings – and their complicit environments and digital extensions, com- more. This includes building some of the
King Uther.
Nimue draws strength from the magical
sword she possesses, imbuing her with DNeg built a large set extension for Hawksbridge port, including the sea.
amazing fighting skills. Everyone, however,
wants to possess this weapon, known as the
“Sword of the First Kings,” as it is said that
whoever wields this Sword of Power shall
be the one true king. King Uther Pendrag-
on wants it, as do the Paladins. Cumber
the Ice King covets the sword, as does the
ruby-eyed shadow lord King Rugen, the
Leper King. At one point Merlin possessed
it, or more accurately, it possessed him,
turning the magician into a primal, vengeful
being. Nimue’s mother relieved him of the
sword to save his life, but he lost his magical
powers as a result. Now, believing the sword
is cursed, Merlin intends to melt it using
Nimue and Pym approach the CG-augmented town of Hawksbridge.
remnants of the long-ago Fey forge fires
from which it was crafted – fires kept in a
treasure room at Rugen’s lair.
The weapon, it seems, is the ultimate
double-edged sword for Nimue, as well.
It enables her to protect the mystical Fey
Images ©2020 Netflix

beings against the murderous Paladins, but

it also begins to nurture a darker side – just
as Merlin had warned.
Season 1 of Cursed, which comprises
10 one-hour episodes, dropped July 17.

e d i t i o n 2, 2020 c gw 9

CGW_Issue2-2020.indb 9 8/7/20 8:12 AM


Nimue entangles a Paladin in practical vines interwoven with CG versions.
DNeg built and animated a demon bear that attacks the young Nimue.
more extensive environments, such as the
town and port of Hawksbridge, where Nimue
tries to catch a boat while fleeing from her
village, but meets Arthur instead.
“It involved a lot more work than what you
see,” says Houghton, describing the environ-
mental extension for Hawksbridge as quite
an undertaking.
As Houghton explains, first, the locale
was surrounded by trees that all had to be
removed in comp, with DNeg building out
the town and port from the existing set.
Using art department drawings and period
reference, the studio added CG churches,
houses, huts, a bridge, and a plethora of
objects, such as carts and wagons, barrels,
fish netting, and so forth. As the village
set was built nowhere near water, DNeg
further crafted moving boats and dynamic
sails, as well as a jetty at the port, and even
added the sea itself. Everything within the
surrounding harbor wall was a practical
set with a large 40-foot-high bluescreen
where the sea was to be. DNeg added the
town beyond the wall, enhanced the set,
extended the jetty by 90 feet into the sea,
and added the water and majority of the
ships and boats.
“The most difficult aspect was getting the
10 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 10
layout right for all of the various angles, be-
cause we started off with the widest angle
of Nimue and [her friend] Pym approaching
Hawksbridge on horseback, which shows
the spread of the town. But then there
are other scenes that feature that same
environment and were filmed from several
different angles,” says Sulé Bryan, DNeg VFX
supervisor on the series along with David
Sewell. “We had to make sure we had the
correct layout to keep all those individual
shots composed correctly so we didn’t have
to tweak each one individually.”
“On one side we had to replace fields
of trees with the city beyond the set wall,
and on the other side, the sea port side, we
replaced a huge bluescreen with an ocean,
boats and ships, and so on,” he adds.
For the construction, the artists used
Autodesk’s Maya for modeling, Adobe’s
Substance Painter and Pixologic’s ZBrush for
texturing, Isotropix’s Clarisse for rendering,
SideFX’s Houdini for effects and simulations,
and Foundry’s Nuke for compositing.
DNeg also built the large set extension for
the village of Gramaire, the seat of power
for Arthur’s uncle, Sir Ector – a site that ap-
pears in several sequences throughout the
series. According to Bryan, this environment
extension was more straightforward but still
required a high level of detail. As was the
case with Hawksbridge, Gramaire required
the artists to remove many trees from the
set, but not all of them – “these trees really
became an issue,” he adds. Similarly, the art-
ists built an environment extension beyond
the limits of the physical set, which included
churches, towers, rooftops, and so forth.
The sets for Gramaire and Hawksbridge
consisted of the large town squares, a
number of side streets, city gates, and the
port area of Hawksbridge. DNeg extend-
ed the towns above the nearest rooftops;
they also created a vista of the entirety of
Hawksbridge as seen when Nimue and Pym
first approach and for the distant Gramaire
as Arthur rides towards the town. “Like
Hawksbridge, [Gramaire] is a location that
is used several times across the series, and
had to remain consistent while being used
again and again,” Bryan notes.
This build included construction of
Ector’s castle by DNeg, as well – an all-CG
set seen in both the background and in
close-up. Achieving the right scale for the
castle was difficult because it was always
visible in shots, which at times required
some slight cheats.
An especially complex environment build
was required for Nimue’s village, which is
sacked by the Red Paladins, who slay or burn
the Fey inhabitants, along with the village
itself. “Our brief was to convey an engulfing
sense of death and destruction,” says Bryan.
To this end, DNeg had to greatly enhance
the amount of flames as only minimal SFX
flames where allowed at the location, and
added other effects such as ash and smoke
throughout the sequence, where the shots
ranged from wide to close-up. The artists
also added 2D crowd elements shot against
bluescreen to increase the number of Pala-
dins and villagers seen in some shots.
In a 70-second Steadicam shot, we see
Nimue running through the village during
the chaos, trying to reach a young Fey boy.
“There’s nowhere to hide with a shot like
this. Usually when you have this sort of
shot, you can get away with certain things
in between the cuts. But when the shot
is over a minute long from start to finish,
everything has to work perfectly,” explains
Bryan. “It’s probably the equivalent of about
15 visual effects shots all in one. There are
multiple huts on fire, the added effects of
8/7/20 8:12 AM
 ash and embers, and additional people.”
The artists also added a huge 180-degree
mountain range across from the lake that
edges the village, which is visible in this se-
quence as well as earlier ones. “Not only did
we have to create this large mountain range,
but we also had to get it into the right angle
for all the sequences, especially the long
shot, where the camera is constantly panning
backward and forward,” explains Bryan.
Of course, there is a great deal of blood
in this scene and others. “We never got to
the point where [the client] said, ‘There’s
too much blood.’ We really had to up the
violence ante,” Bryan notes.
Following the massacre at the village,
Nimue finds her mother, the high priestess
Lenore, gravely injured inside a cave. She
instructs Nimue to take a sword to Mer-
lin. Thus begins Nimue’s true journey.
Since she was young, Nimue has had the
ability to call on the Hidden. In the series,
she uses this ability to entangle enemies in
thick vines, roots, and branches, some of
which are practical and interwoven with CG
versions animated by DNeg. In one particular
sequence, a Paladin is encased by creeping
vines. He is cut down by his brethren and
taken to an abbey to be cared for, where
vines are seen moving in and out, and under,
his skin.
In the scene, the vines pulled from the
Paladin’s mouth were practical elements
shot separately and combined with the
Freefolk crafted the leafy design that appears on Nimue’s face when she is empowered by magic.
CGW_Issue2-2020.indb 11
live-action, and the vines that writhe on the
Paladin’s chest were animated entirely in
FX with a procedural noise texture, whereas
most of the other [vine] animation in the
series was done in Maya, says Houghton.
All her life, Nimue has been taunted for
the strength of her magical powers and
considered an outcast by her peers and the
elders, stemming from a demon-bear attack
when she was a child. She is saved by the
Hidden, but not before she is “marked” by
the bear across her back, leading the villag-
ers to believe she is cursed by dark spirits.
DNeg artists crafted the CG bear based on
reference from Houghton and Miller, and
choreographed the attack. As reference,
the artists examined footage of large brown
bears, studying their gait, stance, and the
way they shift their weight.
But this is no ordinary bear. “Frank wanted
a specific style for this creature, so we
had to obtain the right balance between a
demonic bear and a real-world bear, which
ended up being a 20/80 split [respectively],”
says Bryan. Moreover, the animal was 4 feet
taller than a typical bear, at about 12 feet
high. “It had to tower over her and overex-
tend his jaw, enough to eat her head and
torso in one.”
Freefolk: Nimue’s Facial Vines,
Creatures, Fey Camp
While DNeg created the creeping vines in
the forests, Freefolk generated the so-called
Fingers of Airimid, a design from leaves and
thorns that appears on the skin of Nimue
and the Fey elders when they are empow-
ered with magic or are angry and in danger,
respectively. “Part of the challenge is the
way the effect comes on through the skin,
almost like a blush,” says Meg Guidon, exec-
utive producer at Freefolk. “This is important
because it is an emotional response in the
Fey characters, betraying their special pow-
ers. It is magic but has to look organic.”
According to Harin HIrani, Freefolk head
of CG, the group explored multiple avenues,
including an effects approach using Houdini,
to create a textural pattern that grew over
a 3D model of Nimue the studio had made.
Once they had settled on an effect they
liked, they streamlined the process in order
to deliver the effect across many shots
and to easily alter the timing. In the end,
they ported the texture growth pattern into
Nuke, where they could control the growth
and timing on a per-shot basis, and then
generate maps that could be applied in the
studio’s 3D software to render displaced ge-
ometry lit for the scene. These CG elements
were then integrated into the live-action
plates using Nuke.
“The biggest challenge with the facial
vines was the tracking process. We used
different techniques on a per-shot basis de-
pending on Nimue’s actions. Sometimes we
could purely use vector tracking inside Nuke,
other times we needed a geometry track in
e d i t i o n 2, 2020 c gw 11
8/7/20 8:12 AM
 with CG Moon Wing creatures that were
falling to the ground engulfed in flames,”
says Guidon. “This included animating one
Moon Wing being executed by the Weeping
Monk; all the pleading and dying actions had
to be retrofit to the timing of the existing
greenscreen performance of a sword slash
by the monk.”
Freefolk extended the cave setting for the Fey refugee camp. Freefolk also created the menacing black
spider creature that enchants Morgana as
she retrieves the sword, which Nimue has
tossed into a vast chasm out of frustration.
Furthermore, the team created a realistic
fawn, shown in close-up. Because the team
was unsure how much movement would be
required, they proceeded to build a muscle
system for the animal in Houdini should it
be needed.
Rounding out the studio’s work were
atmospherics, including a magical storm
summoned by Merlin, a scene requiring a CG
set extension of Pendragon Castle to add
the clifftop location and a wild, crashing sea
at its base.
Milk built the subterranean environments of the Leper King and his treasures.

Goodbye Kansas: Castles

One of the more visually stunning all-CG
[The Pixel Farm’s] PFTrack, and other times “We needed to build an entire burning for- environments was that of Pendragon Castle,
a combination of both,” says Hirani. est around him, and populate the treetops handled by the team at Goodbye Kansas
Freefolk, tasked with 200 shots, also
designed and built the convent where the
dying root-wrapped Paladin was taken, Goodbye Kansas built Sir Ector’s sprawling castle.
compositing the building to sit seamlessly
into the wide establishing shot, which was
filmed on location.
In addition, the team created set exten-
sions for the Fey refugee cave, where the
magical folk hid from the pursuing Paladins.
The artists at Freefolk designed, modeled, lit,
and textured the upper levels and wider set
extension, which provide scale to the live-ac-
tion set at ground level. “Having created the
upper stories of the camp, we then populat-
ed the rock faces of the camp with crowds
and added animated CG Moon Wings flying
in the air above the camp,” says Hirani. The studio also built Uther Pendragon’s stylized castle.
In an early scene after the druid village at-
tack, the studio built an all-CG wall of flames
and forest through which strides the feared
Weeping Monk, who is hunting the magical
beings for the Paladins. Meanwhile, Moon
Wings fall from the trees and burn around
him. This was a 40-second continuous take
of the monk walking to the camera, shot
against greenscreen, with extensive effects
elements simulated in Houdini.

12 cgw e d i t i o n 2, 2020

CGW_Issue2-2020.indb 12 8/7/20 8:13 AM

 Arthur and the Green Knight escape the CG flames as Paladins burn the windmill, destroyed by Mr. X.
under the supervision of Ditch Doy. This
included exterior shots of the castle perched
on a ledge and the vista surrounding it, as
well as lavishly dressed interior set exten-
sions. As the series is based on the classic
Arthurian legend, albeit from a new angle,
the artists at Goodbye Kansas made a lot
of nods to the medieval era as well as to the
classic story of Arthur in the castle’s design.
As such, most of the work was based on art
developed in-house with Raf Morant, head
of art department, in conjunction with the
showrunners and Houghton.
“Pinning down the look of Pendragon
Castle was probably our biggest challenge,”
says Doy. “As the seat of Uther’s power in
the show, it had to evoke his character.”
Initially, the design was based on tradition-
al medieval castles in Wales, but as the
concept evolved, it moved more into the
realm of fantasy as the artists worked with
Miller, who contributed a graphic, stylized
aesthetic and influences from German
Expressionism. Ultimately, the architecture
returned to a more reality-based look, yet
still maintained a lot of Miller’s vision.
Goodbye Kansas approached the build in
a modular fashion, constructing a medie-
val kit of parts that could be modified and
re-purposed. This enabled the crew to
CGW_Issue2-2020.indb 13
match the designs while also dressing the
castle as the story required it. For modeling
and animation, they used Maya, as well as
Houdini for simulations and Chaos Group’s
V-Ray for rendering. Compositing was done
in Nuke.
While Pendragon was the studio’s main
asset for the series, the team also created
the interior throne rooms for both Sir Ector’s
and Uther’s castles, and extensions to the
exterior of the executioner’s courtyard. “As
we had created a lot of parts in the medieval
style for our Pendragon build, additional top-
ups such as those were quick to incorporate,
and added some additional grandeur to the
scenes,” says Doy.
Collectively, the studio handled 126
VFX shots.
Mr. X: Windmill Siege,
Battle at the Beach, the Widow
When it came to Cursed’s big battle scenes,
Mr. X was in the thick of it. This included
the windmill siege when the Paladins trap
Arthur, Gawain the Green Knight, and the
Fey inside a windmill and attempt to burn
them out. Among Mr. X’s collective 326 VFX
shots were those pertaining to the large-
scale battle on the beach between the Fey
and tusked people, against the Paladins and
armies of Uther and the Viking Ice King. In
addition to these two main set pieces, the
studio crafted a number of shots scattered
throughout the last six episodes, including
the all-CG character the Widow.
The lower half of the windmill was
practical, while Mr. X digitally extended it,
adding an extra set of vanes that were later
destroyed by CG fire, crafted in Houdini.
“It had to be a seamless match,” says Tim
Stevenson, VFX supervisor at Mr. X, of the
digital build. “There were a lot of drone
shots, moving shots, handheld shots, and
they all had to be tracked to where there
was no distinction between the top and
bottom portions of the windmill.”
Moreover, the CG fire was also mixed with
practical effects and had to seamlessly blend.
As the structure burns in the sequence,
more and more of the fire becomes CG until
the roof collapses, at which point the fire is
all-digital, as is the windmill itself. “Before
we destroyed it, we had to build a full-digital
version of the practical windmill – from the
cloth textures to the wood texture, it had to
be a one-to-one match,” says Stevenson.
Eventually, the heroes and Fey file out of
the inferno, while Nimue, who is outside,
manipulates the flames and turns them into
a giant CG vortex that rises and blows back
8/7/20 8:13 AM
 Nimue battles five CG wolves in a CG environment, crafted by Milk.
on the Paladins, burning them and creating
cover for the Fey group. The vortex had to be
believable yet still have a magical component.
“Fire is always difficult; there are scalabil-
ity issues and dynamic issues, especially
when you are combining digital fire with
live-action fire; there’s a believability factor
that has to be addressed,” says Stevenson.
“We had some practical fire in the shots
that we could match, but there was very
little in terms of the forgiving nature to that
sequence because it was filmed in broad
daylight, one of the more difficult scenarios
for doing believable fire.”
For the epic battle on the beach in the
last episode, Mr. X used Massive crowd
simulation to turn 100 extras into 500 –
Vikings, Fey, Tusks, royal soldiers – according
to Houghton. The sequence had a number
of high-angle shots looking down onto the
beach that were enhanced with digi-double
crowd sims generated in Massive. When
on the same level as the action, Mr. X used
elements of Fey and Vikings fighting, shot
against bluescreen, to increase the numbers
in the battle sequence. “The choreography
was the hardest part because you are deal-
ing with so many shots of people in close
combat and wide-angle shots, so planning
became crucial,” adds Stevenson.
Aside from the battles, Mr. X also created
the all-CG Widow, a reaper-looking charac-
ter that Mr. X helped design and build. Ini-
tially, she was to be an actor with prosthetic
make-up, but she wasn’t scary enough for
Wheeler and Miller, so the artists crafted a
computer-generated skull-like face that is
less rigid and expressive.
Milk VFX: Wolves, Culzean
Castle, Caves, Waterfall
Milk handled a variety of work on Cursed,
crafting 390 shots encompassing a pack
14 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 14
of CG wolves developed from concept
through to animation, as well as a forest
environment in which Nimue fights with
them; and a dramatic waterfall environment
where a combat sequence takes place at
the climax of the final episode. Milk’s brief
also included a number of environments: the
CG Culzean Castle and subterranean world
environments for the Leper Kingdom; the
underground chasm at Cailleach; digi-dou-
bles of heroine Nimue and the Paladins for
the waterfall sequence; the green Fey forge
fire; and the glow and effects on the sword.
The studio also created additional environ-
ments and effects, including chopping off
the hands of the character Bors.
In one revealing scene, the sorceress Yeva
casts an incantation that enables Nimue
and Merlin to virtually meet for the first time,
at Culzean Castle. Here, Nimue is given a
look at the past and at Merlin’s connection
to the sword, leaving her unsure whether to
follow her mother’s dying instructions.
The sequence was filmed among the
ruins of Waverley Abbey in Surrey, which Milk
developed into a larger, grander build of a
castle city ruins. “There was only a little bit
of wall standing [on-location]. Most of the
castle was a CG set extension,” says Hough-
ton. Therefore, the look and textures of the
digital castle needed to match the unusual
texture of the existing monastery ruins that
appear in the plates.
The challenge was building an environ-
ment that could sit in many different shots
and angles without Milk having to make be-
spoke separate builds, according to Ciaran
Crowley, VFX supervisor at Milk.
Milk additionally built the subterranean
environment for the Cailleach, an evil-
filled underground chasm where Morgana
encounters the Freefolk-made spider while
retrieving the sword for Nimue. Milk further
Culzean Castle’s CG set extension.
built the subterranean environments of the
Leper King, which was mostly filmed on a
small set and then digitally extended. “These
were quite challenging to build, as we had to
integrate some floating set walls within the
design of the environment,” says Crowley of
the Leper King caves. “The throne room was
imagined as a vast cathedral with a touch
of backlighting – with a direct reference to
Frank Miller’s comic-art style.”
Meanwhile, the artists filled the treasure
room with gold coins and treasure spilling
out all over the chamber, along with coffins
and copious riches. They also crafted the
green Fey fire, which Merlin steals.
According to Crowley, the most challeng-
ing aspect of the overall job was the wolf
scene, wherein the Paladins dispatch five
(CG) wolves to hunt down Nimue as she is
carrying the wrapped sword to Merlin. The
beasts trap her, and during the fight, she
climbs atop a rock and uses the sword to
dispatch the beasts one by one, behead-
ing three – one in spectacular Frank Miller
style. Like the wolves, the environment in
the scene is all-CG, with the exception of
the actress and rock, which were filmed on
greenscreen.
“The wolf sequence was very tough. It
was shot in the studio, and just prior to the
scene, Nimue is running through an actual
forest and comes into a clearing, which was
essentially a greenscreen studio space with
just the rock in the middle,” says Houghton.
“Milk had to then add the whole of the forest
environment around it.”
The Milk team designed the look of the
wolves from scratch, while the lighting style
and atmosphere they incorporated into
those concepts (which included backlit
lighting, driven from Miller’s signature comic
style) was subsequently used as lighting ref-
erence by the DP and director on the shoot.
8/7/20 8:13 AM

The team referenced Frank Miller’s wolf
from his comic “300” and created concept
designs for several types of wolf, in both
naturalistic and fantasy styles, with the final
creatures having a more realistic look but
are larger than a typical wolf to make them
more menacing.
The artists began crafting the beasts by
first researching wolf fur textures, coloring,
and behavior. Sam Lucas, head of modeling,
built the wolf models in Maya, then used
ZBrush for the finer details around the paws,
face, and mouth, as well as for the muscle
definition. The group used Peregrine Labs’
Yeti for the groom, giving the artists flexibility
to play with the look and feel of the fur, and
Ziva’s rigging and animation software for the
muscle systems.
“The alpha wolf needed to be white; the
second in command, black; and the other
three variations were selected from real-life
wolves by the client – these were gray and
two variations of brown,” explains Crowley.
“Since wolves’ fur color is typically made up
of multiple shades, we incorporated three
color maps to drive the initial variations for
one wolf’s fur, using root, mid, and tip color
maps that would stipulate how a single hair
strand would be shaded. We also added
randomized mutant hairs and melanin in the
shader to push the individual looks. From
there, we also had to make two grooms for
each wolf: a dry groom and a wet groom.”
As Crowley points out, it was imperative
that the fur looked and flowed naturally with
the dynamics. Likewise, it was important
that the bulk and mass of these huge beasts
were properly portrayed in the renders with
the multiple lighting setups.
Milk’s culminating sequence in Cursed
has Nimue, Merlin, and Morgana trying to
escape the Paladins. As they are crossing a
narrow footbridge above a huge waterfall,
CGW_Issue2-2020.indb 15
Artists at Milk crafted this extensive environment, including the CG waterfall.
Nimue is shot with an arrow, then another,
eventually tumbling from the bridge into a
pool of water far below. Not only did Milk
create the waterfall and the environment
around it, but also the digital takeovers
required in the scene.
“It is a big drama scene in a tiny space,
which made it more challenging – with most
of it completed and delivered in lockdown,”
says Crowley.
The sequence was shot outside on a
backlot against bluescreen. The set build
consisted of the bridge, with everything
else created digitally, including the waterfall,
digi-doubles of Nimue and the Paladins,
the rain, and Merlin’s magical lightning bolts
from the storm he whips up as he slowly
regains his power.  takeovers in several of the shots with added
The artists developed the 3D asset of the
environment, while the FX team created
the water simulations and the waterfall (in
just seven weeks during lockdown), as the
latter was broken up into several layers of
simulation, including water surface, foam,
spray, and mist.
For the digi-doubles, scans of the actors
playing Nimue and the Paladins in the scene
were provided by production and rigged
and groomed with cloth sims applied to the
clothing. These were used in wide shots and
for takeovers when the characters fall into
the water.
“We wanted to cover all bases in terms of
character detail, and what helped us mas-
sively was having a full-3D scan of Nimue
for our assets team to begin modeling her
digi-double. All of the base mesh was done
in Maya, then we moved to ZBrush to get
all the extra detail in the skin and clothes.
The textures were made in Mari and were
either hand painted or derived from on-set
element reference, with look-dev done in
Maya using [Autodesk’s] Arnold,” Crowley
explains. “We then passed the character
on to our creature FX team, which handled
all types of character deformation, such as
cloth and hair simulations; the groom for
Nimue was achieved using Yeti.”
In terms of Nimue’s fall, actress Katherine
Langford was shot against bluescreen on
the backlot on a harness with crash mats;
Milk removed the rigs and added the CG
waterfall and the CG set extension at the
bottom of the waterfall. In addition, the
studio did some digi-double takeovers for
the dramatic point when Nimue loses her
grip on Merlin’s hand and plunges down into
the mist. The artists also made digi-doubles
for the Paladins who are killed and fall from
the bridge during the fight, allowing CG
blood splatters in the vein of Frank Miller’s
comic art. 
Ultimate Cliff-hanger
Season 1 ends with the waterfall sequence,
but is it the end of the series? Many ques-
tions were answered and issues resolved,
but others remain. Reviews have been
positive, leaving the door open to a Season 2
if Netflix chooses to open it.
“The way Frank and Tom told this story is
very interesting. Initially, the characters feel
unfamiliar to us in terms of the Arthurian
legend, but gradually the pieces come
together as you start to realize who the
characters are within the existing mythology,
and that’s very intriguing and exciting, and
opens the door for a much bigger story to be
told,” says Houghton.
As for the magic, that was contributed
by the visual effects artists, modern-day
“magic folk.” 
Karen Moltenbrey is the chief editor of CGW.
e d i t i o n 2, 2020 c gw 15
8/7/20 8:13 AM
 Kitbull Purl

Igniting
Passion
PIXAR’S SPARKSHORTS PROGRAM GIVES EMPLOYEES THE
CHANCE TO BRING THEIR OWN ANIMATED SHORT FILMS TO LIFE
Smash and Grab BY KAREN MOLTENBREY

Wind
16 cgw e d i t i o n 2, 2020

CGW_Issue2-2020.indb 16 8/7/20 8:13 AM

 I
t’s not easy to take creative risks when
there’s so much at stake and so much
to lose.
Take, for example, Pixar Animation Stu-
dios, known for its animated blockbusters
that indeed are very expensive to make but
have been met with critical acclaim and
huge box-office success. But, how do you
keep the spirit of taking risks alive in an en-
vironment like this? Such was the question
posed by Lindsey Collins, VP of development
at the studio, to the executive team some
time ago as part of a broad conversation.
Out of this desire to innovate and
challenge themselves, Pixar’s SparkShorts
program was born.
SparkShorts gives people within the Pixar
community the opportunity to take creative
risks that they are unable to take while
working on a feature film. “There are people
here who we’ve had our eye on and thought,
what if we gave them a relatively small
amount of money and put some incredibly
talented and creative people around them,
and have them work with little oversight,
to see what they would make, what stories
they would tell?” says Collins, who heads
up the program. “Would we be surprised,
and would it allow us to take some risks in a
relatively safe way?”
Initially, there was no distribution model
for the shorts in the program. “Maybe that
was the safest way for us to really let them
do what they want to do and not feel like we
have to be looking over their shoulder all the
time, making sure that it’s perfect,” Collins
says. “But deep down, we knew that if what
they made was great, we wouldn’t be able
to help ourselves from wanting to get [the
projects] out into the world and would figure
out how to do that. Nevertheless, it was a
leap of faith but within a safe environment
that forced us to take some risks.”
Pixar has been long revered for its theatri-
cal short films, which have been associated
with a feature release. Alas, under the theat-
rical model, the short filmmakers no doubt
felt pressured to make something that was
Pixar-branded and worthy of this presen-
tation method, Collins believes. Thus, they
felt the short had to look as perfect as the
feature film it was released with, and often
second-guessed some of their instincts and
followed less-risky paths, and the produc-
tion process became lengthy and expensive.
“We wanted to give more people oppor-
CGW_Issue2-2020.indb 17
tunities and see what stories they would
tell, so we tried to release them from the
pressure of theatrical shorts,” explains Col-
lins. “And every time the temptation is there
for us to put ‘Kitbull,” or ‘Float,” or another of
the films in front of a Pixar feature, we have
to talk ourselves out of that. We have been
down that path before, and this [Spark-
Shorts] is something different.”
Taking Risks
A few years ago, as Collins stepped off
Finding Dory, which she produced, and into
development, she was able to focus on the
SparkShorts initiative and begin identifying
those within the studio who might be ideal
choices to put into the director and produc-
er roles of these shorts.
With SparkShorts, there is no pitch pro-
cess or approval process. As Collin explains,
everyone spends a lot of time together in
the studio, and she and the executive team
become aware of those individuals on
shows who have risen to the top, or were in-
formed about others who were working on
concepts of their own. The only rules were
that the short had to be animated, that is a
narrative story, and that it is in-line with the
values of the company.
Once selected, the director is given six
months and a modest budget in terms of
people-hours. “It’s such a different pace to
what they are all used to,” Collins says of
the participants. “The creation process here
is usually long, but here they made these
amazing, great, heartwarming, different
stories. That’s been really rewarding for all of
us here.”
The directors/teams are allowed to use
studio resources (equipment and talent).
Often they rely on their own brain trust
– friends and colleagues – to help them
flesh out their ideas, and when the concept
becomes solid and official, they are assigned
a producer and a crew; they interview and
select their supervising TD, giving those per-
sons an opportunity for leadership exposure
as well, through the program. Collins points
out that Pixar has had female supervising
technical directors from the SparkShorts
program who have later assumed supervis-
ing TD roles on feature films.
Collins acknowledges that the process
was difficult for the first few filmmakers
because the program hadn’t been formal-
ized yet, and they were flying under the
radar. As such, not many of their colleagues
knew what they were working on. But when
the fruits of their labor were revealed at a
company meeting, it was to the delight and
surprise of the entire staff. Now, everyone
at Pixar is familiar with the program and
excited about it.
Careful to avoid labeling or defining the
program by its content, the studio released
the first three SparkShorts – “Purl,” “Smash
and Grab,” and “Kitbull,” which are very
different from one another in terms of story
and aesthetic – simultaneously to illustrate
the diversity and breadth of the program.
The three shorts celebrated their world
premiere at the El Capitan Theatre with a
limited showing in January 2019, followed
by debut on Pixar’s YouTube channel. Since
then, the shorts have been played on the
new Disney+ streaming platform.
“We wanted to let the SparkShorts pro-
gram evolve and show us what it wanted to
be,” Collins explains.
So far, seven of the short films have been
released, and as of this writing, two exper-
imental filmmakers are in the process of
working on the ninth and 10th SparkShorts.
“For now, [the program] is still doing what
we want it to do, which is to give people
the freedom and the opportunity to tell
some stories that are continuing to prove
themselves worthy of being told and people
showing they are worthy of the investment,”
says Collins, who remains hopeful that this
program will continue for some time.
Meanwhile, Collins says Pixar is very cog-
nizant of maintaining an equitable balance in
terms of male-female directing and produc-
ing, and to this end, is trying to have a couple
shorts in progress at a given time.
The goal of SparkShorts was to come
away with some fantastic stories. “That is
a huge win, having felt like we might have
identified some of the future storytellers
in our studio,” Collins adds, which is a big
plus, as the current (or traditional) path to
becoming a feature director is a long one
at Pixar, as the moviemaking process is
typically very lengthy. SparkShorts, however,
shortens that path, making it a win-win for
the studio and these hidden gems: the film-
makers themselves and their projects.
To date, the following short films have
been released under the SparkShorts
program: “Purl,” “Smash and Grab,” “Kitbull,”
“Float,” “Wind,” “Loop,” and “Out.” Here
e d i t i o n 2, 2020 c gw 17
8/7/20 8:13 AM
 we take a look at some of them, while the
others are detailed on our website, cgw.com,
in the In Focus section.
Float
(7:04 runtime)
Director/writer: Bobby Rubio
Producer: Krissy Cababa
“Float,” the fourth SparkShorts film released
by Pixar, is a heartwarming story of a parent
and a child who is “different.” It is a story of
many parents who are trying to cope with
such a child, although for “Float” director/
writer Bobby Rubio, it is a very personal one.
It is his and his son’s story.
In “Float,” a father discovers that his son
is unlike other kids in the most unusual way.
To keep them both safe from judgment,
Dad keeps the boy out of sight – but when
his son’s ability becomes public, Dad must
decide whether to run and hide or accept
his son as he is.
“The inspiration for ‘Float’ came from my
relationship with my son, Alex, who is on the
autism spectrum. When he was diagnosed
10 years ago, I was trying to deal with it emo-
tionally and had a difficult time processing
his diagnosis,” explains Rubio.
Rubio, a Pixar storyboard artist, is a
storyteller who makes his own comic books
outside of work. And this story was one that
had been gnawing at him for years. “When I
For director Bobby Rubio, “Float” is based on a very personal story/journey.
18 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 18
was struggling [with the situation,] my wife
said, ‘You make comic books, you tell stories.
Why don’t you make a comic book to help
you work through this?’ ” he recalls. He be-
gan the process of doing just that, but it was
an emotional road for him to travel, making
it very slow going. Determined to complete
the journey, he opted to turn the story into
an animated short film instead.
“I was going to do it in my spare time,
and I drew a storyboard animatic for it and
showed it to my friends, who suggested I
show it to Lindsey Collins, executive pro-
ducer of SparkShorts,” says Rubio “I wasn’t
too sure, as this was a personal project, but
I know her and thought I might as well get
her impression of the story. She loved it and
thought it would be a good candidate for
the SparkShorts program.”
Rubio’s story unfolds as he, in the role of
the father, is playing with his son, Alex, in the
front yard, but when Alex shows his unique
ability to float, he immediately takes his
child indoors, out of sight from the watchful
gaze of neighbors and passersby. He keeps
his child hidden from the world, allowing
him to float indoors. When he finally does
take the youngster to the park, he tries to
prevent Alex from floating by weighing him
down with rocks in a pack attached to the
child’s back. All seems fine until Alex catches
the other kids by surprise by exhibiting his
uniqueness. At this point, dad voices the
only dialog in the film, chiding the child for
not being normal. Shocked at his own out-
burst, dad hugs Alex, realizing his mistake,
and begins pushing Alex on the swings,
bringing happiness to the boy. And then the
boy floats – as his dad smiles.
“Float” is reflective of the Pixar house style.
This became a big advantage when, to save
time and budget, the team used some assets
available at the studio when creating the
short. “We went ‘total independent film style’
and looked at what we could get our hands
on. Toy Story 4 was coming out, and I liked the
look of it. So, we used a bunch of assets, back-
ground characters,” says Rubio. In fact, viewers
may recognize Alex’s house – it was Bonnie’s
house in Toy Story 4. It is just changed digitally
to make it look different in “Float.”
According to Rubio, he wanted ‘Float’ to
have a live-action feel and took a more gritty
approach to reflect the story. “It needed
that,” he says. “It’s about a father who is
experiencing depression, so there are a lot
of dark moments. There are times we get
super dark with the backlight, especially
with sequences in the house. We went as
dark as we could. Michael Sparber, director
of photography, was amazing.”
To this end, Rubio and the lighters looked
to live-action movies for inspiration, and one
scene of the father and son in the house by
the window is in full silhouette – dark with
black around them. The inspiration for that,
8/7/20 8:13 AM
 according to Rubio, was the feature film E.T.
Rubio’s team mainly used tools from the
Pixar box to craft the short. The biggest
technical hurdle, according to the director,
was the child’s hooded jacket when the
hood was pulled over Alex’s head. “We didn’t
have a huge budget, so we tried to cheat as
much as we could. We showed a little bit of
a hand going over the head, but we didn’t go
completely over the head. Also, I wanted the
dad to take off the jacket completely, and
that was difficult. We cut corners for that,
too,” says Rubio. “You see dad start to take
the jacket off and then it cuts to another
shot, and we come back and the jacket is
completely off of Alex.”
Creating the tears Alex sheds after his
father’s outburst was quite the challenge,
as well. But, there were just some elements
to the short that were too important to skip,
despite the difficulty they presented. “They
were really important pieces of the film, and
I am super grateful that Jane Yen, supervis-
ing technical director, and her team were
able to pull them off,” says Rubio.
Completing the short in the given time
period required cutting a full minute from
the film. “That was a tough one. We went
right down to the wire. But I feel like we were
successful,” Rubio adds. “I still miss that one
minute. Maybe there will be a director’s cut
some day,” he says wistfully.
As for the float animation, the timing
couldn’t have been better to tackle that, as
many of the animators were just coming off
Incredibles 2, in which Jack-Jack flies. “We
were looking for more of a graceful Peter
Pan type of gliding across the screen for
Alex, as opposed to Superman’s powerful
directional flight or Jack-Jack’s tumbling,”
Rubio points out.
Despite the emotional toll the film took,
Rubio relishes the opportunity to direct. It
also gave him the chance to learn the pipe-
line, something he never experienced before
as a storyboard artist. “As a director, you get
to see the pipeline from the beginning to
end, from story to character designs, to the
art department and to the sets – how they
are built – and animation,” he says. “This is
a totally different animal from traditional
animation [which he previously did at Dis-
ney].” Plus, there are the music and lighting
elements, too.
“All the way from beginning to end, it
was a big education for me and I thoroughly
CGW_Issue2-2020.indb 19
“Float” looked to live-action films like E.T. for lighting inspiration.
enjoyed it. It was a big learning curve to go
through in just six months, but I am confi-
dent that if I had to do it again, that I could,”
says Rubio.
Loop
(8:00 runtime)
Director/writer: Erica Milsom
Producer: Krissy Cababa & Michael Warch
“Loop” might be a short film but it is long
on emotional content. The animated film,
directed and written by Erica Milsom, tells
the story of two teens at summer camp: a
talkative outgoing boy and a very quiet girl,
both of whom are paired up on a canoe
course. The girl, as we come to learn, is not
simply shy. She is non-verbal, autistic. To
complete their trip, they must learn how
the other experiences the world so they can
communicate.
Milsom already had been making films on
the side and felt participation in the Spark-
Shorts program was a natural evolution to
her storytelling interests.
Milsom is a documentary filmmaker
in Pixar’s Creative Content department,
telling the stories behind the stories of the
studio’s features. So when the SparkShorts
opportunity came up, she was immediately
interested, wanting “to explore working
inside an animated story versus about an
animated story,” she says. (“Loop” aired in
early January, and now Milsom is directing
an upcoming series about the studio that
also will be shown on Disney+.)
Prior to working for Pixar (she has been
there for approximately 15 years), Milsom
worked within the disabled community,
later teaching a class for disabled adults
in her neighborhood where she had met
many people who communicated through
gestures rather than words. Initially, her reac-
tion was to talk her way through a situation
with them; the effect was not a good one,
she adds.
“People were not trying to communicate
with me because I was creating a wall of
friendly chatter,” the director says. “The
inspiration for the short came from the work
I did as a volunteer.”
In the film, the boy Marcus tries to com-
municate with non-verbal Renee by talking
a lot. Renee becomes agitated; Marcus,
frustrated. Eventually, they figure out how to
communicate.
Milsom also drew on her love of canoeing
while crafting the film. “If you are not getting
along or not in sync with your canoeing
partner, you are kind of stuck. You can really
work against each other,” she points out.
“So, I thought this would be a good location
for fun and also some drama.”
“Loop” is indicative of the Pixar aesthet-
ic. Using the studio’s pipeline, the group
called on the animators’ talents to find
very specific and different behaviors and
performances from the characters – for the
animation and acting, as well as in the voice
performances.
The biggest technical challenges, accord-
ing to Milsom, were the visual points of view
and the sonic points of view of the girl, Re-
nee. “We had learned that people with au-
tism have a different sensory experience of
the world, that they might have an amplified
sense of the world around them,” Milsom
explains. “We only had a short time to make
the film and had to define an entirely differ-
ent look for Renee’s point of view.” Danielle
Feinberg, cinematographer, then asked
Milsom to provide some information on her
idea of what this POV was, which Feinberg
then sent out to the lighters, who were not
yet on the project, asking for examples of
different kinds of lighting.
e d i t i o n 2, 2020 c gw 19
8/7/20 8:13 AM

“Loop” filmmakers considered the visual and sonic POVs of the autistic Renee.
“We got five potential looks from people
around the studio. They were amazing. We
shared them with our consultants at the
autistic self-advocacy network to get their
opinions. Danielle and I both liked one in
particular but felt we should choose the one
that feels most appropriate to them,” says
Milsom. The group, in fact, chose a com-
pletely different look, almost the opposite of
what Milsom and Feinberg had liked.
“The one we chose had a de-saturated
look and these beautiful bloom lights, and
they chose something that has a deep
saturation in the center and the lights
bloomed but in a different way, harsh on the
highlights,” Milsom explains. “But they look
beautiful in the film. So, whenever you are
with Renee and looking at the world through
her point of view, the world has this particu-
lar look, this deeper saturation in the center
and bigger highlights on the sides. It actually
amplifies a little if there’s more emotion and
the highlights get even hotter.”
There were other considerations, as well.
Kenny Pickett, sound designer, wanted to
know how Renee would experience sound,
so the sound effects are different from her
perspective than from that of Marcus. It also
differs depending on whether she is calm
Animators used very specific and unique behaviors and performances for the characters.
20 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 20
or overwhelmed. Moreover, the camera de-
partment, led by DP Sylvia Gay Wong, made
sure that when you’re looking throughout
Renee’s POV, eye contact is not made.
“The details and the careful consider-
ation that went into the film are significant,”
Milsom notes.
In fact, the complexity of the film is
apparent throughout, even in the water
and hair simulations. Marcus wears a hat,
which cuts down on some of the sim work,
while Renee was given a hairstyle that was
not overly complicated to simulate. “She is
a young woman of color, so on a technical
level, emulating darker hair is a little easier to
do than lighter hair,” Milsom points out.
Also, the two characters are on water for
a large part of the film, requiring water sims.
Effects lead Kylie Wijsmuller, who was fairly
new to Pixar, figured out the water cycle, the
waves generated by the canoe. “I knew the
water would be difficult, but I hadn’t really
thought about the water reacting on the
sides of the canoe,” says Milsom.
The simulation team further tackled the
fit of the life jackets on the characters and
moving them in such a way that they did not
look too rigid..
“The simulation team was so great,” says
Milsom. “They are all such superstars in this
industry, and their care and attention, and
contribution to the story, is very deep.”
It goes without saying that having the
characters talk (most of the other Spark-
Shorts do not have characters who speak)
presented some hurdles. Not only was it
more work for the animators, but recording
the voice actress for Renee did not follow
the typical Pixar process. With assistance,
the filmmakers found Madison Bandy, who
voices Renee. She is autistic and can be
overwhelmed due to sensory overload.
Therefore, her voice performance was
recorded at her home and captured her
performing naturally – when she was happy,
stressed, and so on.
Milsom credits her team and her produc-
ers for the project’s success, pointing out
that it takes a partnership to make a film like
this, and producers Krissy Cababa and Mi-
chael Warch were always right there. “They
were very much like the strategists. I was the
confidence lady. We’d talk about how great
we were going to make this, but they would
strategize with people and make sure we got
things done. They were truly great partners.”
Milsom steps away from this project with
a newfound appreciation for the animation
process, being inside of it and collaborating
with artists to bring two little somebodies,
as unique people, to life in the animated
form as Renee and Marcus.
“I feel they are really honest and true
and authentic as characters, but they’re
still designed, and their way of being in the
world was carefully considered. It was one
of those amazing artistic experiences that
only comes in animation, because you have
8/7/20 8:13 AM
 “Out” uses a unique painterly style.
to design, consider, discuss, and create
every single part of your story,” says Milsom.
“How does Renee move her hand under the
canoe? How does Marcus flip around that
reed when you’re only looking at his hand
in certain shots? I had no idea that those
would be some of the most challenging
shots to get right, because it was such a
subtle and emotional gesture.”
In addition to coming away with a love
for animation and connecting on a story so
deeply, Milsom also had the opportunity
to learn about autism, which she says was
a treasured take-away. “It’s hard making
films, particularly smaller stories with unique
characters that you haven’t seen before,”
she says. “It was a wonderful journey.”
Editor’s Note: “Loop” recently was named
SIGGRAPH 2020 Computer Animation Festi-
val’s Best in Show.
Out
(9:33 runtime)
Director/writer: Steven Clay Hunter
Producer: Max Sachar
The seventh in the series of SparkShorts,
“Out,” released in May, tells the tale of two
guys and a dog who are packing up to move,
when the parents of one of the men arrive
unexpectedly to assist. The problem is that
Greg’s parents are unaware that he is hiding
a secret: He is gay and in a relationship with
Manuel, and is moving from a small town to
a big city, where he believes his relationship
will be more accepted. Greg quickly and
quietly ushers Manuel out the back so his
parents do not see him. Manuel complies
but wants Greg to tell his parents the truth.
Meanwhile, a cosmic purple cat and pink
dog emerge from a rainbow to watch the
situation unfold, and they enchant the collar
on Greg’s dog, Jim. As the parents make
themselves at home, Jim’s collar magically
attaches itself to Greg, and the two switch
bodies. Jim, now in Greg’s human body, pays
CGW_Issue2-2020.indb 21
When the parents arrive in the short, the “fun” begins.
special attention to dad, while Greg, in Jim’s
dog body, tries to hide a beloved photo of
him and Manuel from his mother, with many
hilarious close calls. Mom eventually pours
her heart out to the dog (who is actually her
son), saying she would like to tell her son that
she and his father hope that one day he will
find someone who loves him as much as they
do, and that he will make their son happy.
Greg and Jim return to their own bodies,
and Greg later introduces Manuel to his par-
ents, who welcome him. As for the cosmic
cat and dog, they return to the rainbow, their
work complete.
The short film is written and directed by
Steven Clay Hunter, an animator at Pixar. “I
wanted to create a story for my closeted gay
self, something silly and funny like the car-
toons I used to watch growing up,” he says.
“I wanted to create myths for LGBTQ kids, to
inspire us and get families talking.”
The story unfolds using a unique painterly
style that is very atypical from what we
are used to seeing from Pixar – yet, that is
exactly the spirit of SparkShorts, to experi-
ment with story and style.
According to Hunter, the Little Golden
Books served as a big inspiration, as the
“Out” crew were looking to create some-
thing that had a bit of childlike wonder to it.
Mary Blair’s concept paintings from Alice in
Wonderland greatly inspired them as well.
Hunter guided his team, averaging a
dozen people at any given time. They utilized
Pixar’s in-house pipeline, using Presto for rig-
ging and character animation, and recycling
as much imagery as they could from the
studio’s feature-film backlots, with sets lead
Kristifir Klein filling in the modeling gaps.
Rendering was done with Foundry’s Nuke, as
Colin Thompson, look-dev lead, worked with
DP of lighting Andrew Pienaar to achieve the
desired lighting and painted brushstrokes.
The biggest technical hurdle, contends
Hunter, was figuring out the look for the
short and how they would achieve it. “We’ve
never really tried to create a moving painting
before at Pixar, and we were still defining
what that would look like while we were
making the film,” he says. He praises Thomp-
son and Pienaar, along with DP of camera
Matt Silas and supervising technical director
Gordon Cameron, as the “brains” behind it
all and keeping the group focused on staying
true to that painted aesthetic.
“In film, we rely on depth of field to com-
municate where the audience’s focus should
be, but we didn’t know what that would look
like in a painted render,” Hunter says. “We
didn’t think we’d be able to figure out how to
do that in our production’s time frame. But
Colin came in one day with a big smile and
said, ‘I think I figured it out.’ And he did!”
Working on the experimental short was a
big adjustment for the team, which is used
to working on big-budget feature films,
for which they are given ample time and
resources to polish their work to perfection.
“But on an ‘indie film,’ you have to learn to
get your strongest ideas out in the shortest
amount of time possible, and then let it go.
That’s an easy thing to say but an even hard-
er thing to do,” Hunter says.
Nevertheless, the rewards were plentiful.
Hunter has helped many a director tell his or
her story but had never sat in the director’s
chair himself before this. “So when a ques-
tion came up, I suddenly couldn’t turn to
the director to find out what to do. I was the
one who had to have the answers!” he says.
“I had to learn what kind of filmmaker I was,
what kind of storyteller I was. And, I slowly
gained that confidence as we made [the
short], but it wasn’t there when we started
the process.”
Hunter especially enjoyed working in the
smaller teams and figuring out how to make
films and still tell great stories, “which is
what Pixar is all about,” he adds. 
Karen Moltenbrey is the chief editor of CGW.
e d i t i o n 2, 2020 c gw 21
8/7/20 8:13 AM
 Universal Scene
Description
PIXAR DOES A DEEP DIVE INTO USD AND ITS EFFECT ON CONTENT CREATION

BY F. SEBASTIAN GRASSIA AND GEORGE ELKOURA 

22 cgw e d i t i o n 2, 2020

CGW_Issue2-2020.indb 22 8/7/20 8:13 AM

 F
or as long as we have been creating
digitally synthesized images, we have
needed ways to describe the 3D scenes
we are synthesizing in a way that is
mathematical enough for computers to
understand, but which is also understand-
able and manipulatable by technologists
and artists.
This article tells the story of some of the
challenges Pixar faced in describing, sharing,
and transporting 3D scenes as our pipeline
evolved over 25 years, from creating the first
feature-length computer animated movie,
to making it possible for our productions’ ar-
tistic visions to keep growing in richness and
collaboration. Thus, we present Universal
Scene Description (USD), Pixar’s open-
source software for describing, composing,
interchanging, and interacting with incredi-
bly complex 3D scenes.
USD is a cornerstone of Pixar’s pipeline
and has been on a rapid and broad adoption
trajectory, not only in the VFX industry, but
also for consumer/Web content and game
development.
BEFORE THERE COULD BE TOYS
When Pixar set out to make Toy Story in the
early ’90s, we had industry-leading, com-
mercially-available products at the front and
back of our 3D pipeline – Alias’ Power-
Animator at the front for modeling, and
Pixar’s own RenderMan at the back. But to
handle the scale of making a feature-length
animation, we needed to invent or improve
a suite of custom tools and data formats for
everything that needed to happen between
modeling and rendering.
“Scale” here impacts several aspects of
production that weren’t quite as daunting
when making Pixar’s earlier projects (short
commercials and effects): number of artists
who needed to be simultaneously working
on the same project, complexity of the envi-
ronments, and number of acting characters,
number of assets, sequences, and shots.
We had already figured out that one
important component to successfully work-
ing at feature-scale was to separate out
different kinds of data into different files/
assets. While “appropriate format” was a
consideration, more important was that by
separating out geometry, shading, and ani-
mation into different files, we enabled artists
of various disciplines to work independently
and safely, simultaneously, and made it
CGW_Issue2-2020.indb 23
While making Toy Story, an issue of scale required custom toolsets.
easier for them to reuse components.
With these and other organic improve-
ments, Toy Story was possible, but also
illuminated many ways in which our pipeline
needed to become more scalable and more
artist-friendly.
THE INDUSTRY EVOLVES &
PIPELINES GET MORE COMPLEX
As more studios began making fea-
ture-length CG animation, more varied and
powerful software became commercially
available, which was terrific because it ex-
tended artistic reach, but it also made pipe-
lines more complicated. The three problems
of scale only grew (and still do!), but now
the industry faced an additional problem of
data interchange, since it was rare for any
one vendor’s product to read data in its full
richness from another vendor’s – some-
times not even products within a vendor
family could speak to each other.
Many VFX studios, including Pixar, built
pipelines around in-house software, but with
entry and exit points for data from many
vendors. Formats for 3D interchange were
available, both open and proprietary, such as
obj, Collada, and FBX, but none could satisfy
the goal of rich (in terms of schema and
kinds of data), universal interchange.
Studios developed different strategies to
deal with interchange; Pixar built a “many
to many” conversion system called Mojito
that defined a core set of data schemas
into which you could (in theory) plug any
DCC in as an input and get out a data file for
any other DCC (or renderer, though we only
ever implemented support for RenderMan).
Mojito allowed us to adapt to new software
in a reasonably modular way, but it was
expensive to maintain and did not help with
any of our scale problems.
Finally, in 2010, the industry received a
quantum leap forward on the interchange
problem, when Sony Pictures and Industri-
al Light & Magic released its open-source
collaboration, Alembic (see “Share and Share
Alike,” CGW, Summer 2019 issue). Alembic
was designed with rigor and deep knowledge
of VFX pipelines to address the interchange
of “baked” (or time-sampled) geometric
data, with vetted schemas for geometric
primitives and transformations, and a data
model that abstracted “file format.”
That abstraction was critical because
it allowed Alembic to gradually deploy
formats like Ogawa that addressed some
of the important scalability issues for
VFX, such as being able to store massive
amounts of animated data in a file, while
primarily paying the cost (IO and network
traffic) for only the pieces of data a partic-
ular application/node needed for its task.
Alembic arrived (not coincidentally) at a
time when many studios were lighting in
different applications than they were rigging
and animating, so its “bake out the rigged
animation into time-sampled mesh data”
approach was exactly what was needed.
To keep the scope and mission of the
project tight and manageable, Alembic
stayed focused on interchange of “flat” data,
free of any concerns of composing multiple
Alembic archives together, proceduralism,
execution, rigging, or even a run-time scene
graph on which such behaviors could be
layered. Within several years, Alembic had
deeply penetrated the VFX industry, and the
e d i t i o n 2, 2020 c gw 23
8/7/20 8:13 AM

Finding Dory became the first feature film with a USD-based pipeline.
majority of commercial 3D vendors support
Alembic import and/or export.
Pixar did not adopt Alembic in its pipeline,
primarily because from 2008-2010, we had
been developing our own time-sampled
interchange format, called TidScene, with a
slightly different approach to the problem…
but we’re getting a little ahead of ourselves.
REFINING WORKFLOW AND
‘COMPOSITION’
At around the time Ratatouille entered
development, Pixar was growing hungry for
revamping its toolset, which was beginning
to show its age; not only was our anima-
tion and lighting system nearly impossible
to adapt to the multi-threaded future of
performance, but our pipeline had also
grown complicated. Any given shot relied
on dozens of different file formats and
data abstractions: We had one format for
“primary” geometry, but another for UV sets
and other primitive variables that were too
big for the primary format, and a third for
simulation-generated geometry overrides.
We had one format and language for shad-
ers, another for shading networks, and yet a
third to describe how materials were bound
to geometry. It was time to reconsider our
tools, workflows, and pipeline.
The Menv30 project was kicked off, later
renamed Presto when released. Menv30
was an opportunity to take all the learnings
of the earlier toolset and develop them
into a cohesive and well-structured set of
libraries. Perhaps most importantly, the
project sought to represent all of our work-
flows using a single file format, menva, in
a single application that could easily allow
artists to transition back and forth through
a fluid and malleable pipeline. We started
by targeting rigging, animation, and layout,
24 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 24
later adding simulation workflows.
To Presto’s scene graph library, we added
and refined all the features that were nec-
essary to bring together all the disciplines
and workflows that we had so far accumu-
lated. The most fundamental feature was a
layering system that is often compared to
Photoshop’s image layering paradigm. Ref-
erencing, inheritance, and variants complet-
ed the initial set of composition operators
that formed a robust composition engine,
and which, unbeknownst to us at the time,
began the underpinnings of the technology
that would later become USD.
We had initially targeted Toy Story 3
(and missed it) for early adoption of this
brand-new ambitious system and knew it
was going to be difficult to achieve. In order
to make it more tenable, we stopped short
of adding support for the back end of the
pipeline: shading, lighting, rendering, and so
on. That was going to be left to the existing
systems. That meant: interchange.
We now needed a way to translate data
born in Presto to the existing back-end
systems. In order to do that, we invented a
new scene transportation system called Tid-
Scene. TidScene was built using Berkeley DB
and was primarily targeted toward efficient
transport of time-sampled binary data and
rapid preview imaging. As we began deploy-
ing TidScene in more parts of the pipeline,
we kept encountering the need to add more
and more features to it, such as referencing
and inheritance.
Having TidScene meant that we were
implementing things twice, as we kept
wanting to add more features that we had
already implemented in Presto. Worse, the
implementations grew inconsistent and
incompatible with one another.
This became a significant enough prob-
lem that a solution was indeed needed.
The solution? USD.
WHAT IS USD?
USD began as an experiment in 2012 to see
if we could take the low-level data model
and “composition engine” at the heart of
Presto, where it served a rich scene graph
and anim curve-driven execution system,
and build atop it instead a light, highly scal-
able scene graph that practiced lazy data
access, embraced time-sampled data, and
facilitated the creation of “schemas” (think
Mesh, Xform, Material, etc.) to foster robust
scene interchange.
Our proof-of-concept came very close to
the performance of TidScene on equivalent
large scenes, while opening the door to the
full suite of Presto’s composition operators
(though we chose to leave out a few of USD
for performance and/or complexity reasons).
However, we believed we needed to be much
more than an order of magnitude better than
TidScene to satisfy the growing needs of the
studio for the next 10-plus years.
We made good progress toward that goal
by making data structures thread-safe/
efficient and algorithms multi-threaded at
all levels of the USD software stack. The
development of “native scene graph in-
stancing” to augment schema-based point/
array instancing allowed us to load scenes
containing millions of primitives in seconds
on modern workstations.
Performance has consistently been a top
priority for USD, and we continue to make
small and large improvements. But while
performance is a key “wow factor” for USD
that facilitates artists’ ability to iterate, there
are several other important reasons why USD
has been so quickly and broadly embraced.
COMPOSITION ALGEBRA
If there is any magic in USD, it is the “com-
position engine” that provides the ability to
combine “layers” worth of scene description
in many useful ways, and lets users make
non-destructive edits. The rules by which
this powerful combinatorial engine works
have been refined over several different in-
carnations, starting with work pioneered on
A Bug’s Life, and protected by more than a
dozen patents that Pixar released for public
use as part of open-sourcing USD.
Understanding the full implications of
the composition rules can be daunting, but
8/7/20 8:13 AM
 simple uses are easy to understand, and
the complete behavior can be described
as the recursive LIVRPS (pronounced “liver
peas”) algorithm, documented on openusd.
org. We refer you to the website for deeper
exploration of USD’s composition arcs, but
we will describe several of them here as
they pertain to addressing the scalability
problems we discovered way back on Toy
Story. Note that each of the composition
arcs can be used to solve other problems as
well, and one of the things in which we have
invested heavily is trying to make sure that
any possible combination of composition
arcs behaves in reasonable ways.
Layering for Multi-Artist Collabora-
tion – Layering in USD is similar to layers in
Photoshop, except that in USD each layer is
an independent asset, and often, each layer
will be “owned” by a different artist or de-
partment in a pipeline. While the modes by
which data in different layers can be merged
together is much more restricted in USD
than it is in Photoshop, layer compositing al-
lows multiple artists to work simultaneously
in the same scene without fear of destroying
work that another artist is doing.
One artist’s work may “override” another’s
because their layer has higher precedence
than the other artist’s, but since layer prece-
dence in a sequence or shot in a CG pipeline
often corresponds to stages of the pipeline,
this is not often a problem. When it is, the fact
that each artist works non-destructively with
respect to other artists’ work means we can
deploy a range of tools to handle situations in
which something unexpected happens.
Referencing and Variant Sets for Asset
Complexity – Referencing in USD is not dis-
similar to referencing features in packages
like Autodesk’s Maya, but more powerful
in that careful consideration has gone into
specifying how multiple references can
combine as we build up assets, aggregate
assets, sequences, and shots via chained
and nested application of referencing.
References allow us to build simple or
complex “component” assets out of mod-
ular parts. References also allow us to build
up environments out of many references
to individual component assets – some of
the costs will be shared at run time among
many references to the same asset, and
when we enable those references to be
instanced, that sharing increases substan-
tially more.
CGW_Issue2-2020.indb 25
References allow us to bring environ-
ments into sequences and shots. At all
“levels of referencing,” USD allows you to ex-
press overrides on the referenced scenes in
a uniform way, so once you learn the basics
of “reference and override,” you can use it to
solve many problems.
One significant source of pipeline
complexity is the number of “variations” of
published assets that are typically required
to provide the level of visual richness that
audiences expect from modern CG films.
USD provides variant sets as a composition
arc that allows multiple different versions of
an asset to be collected and packaged up
together, providing an easily discoverable
“switch” for users of the asset to select
which variation they want. This switch
(called a Variant Selection) is available in the
USD scene regardless of how “deeply” (via
referencing and layering) it was defined in
the scene’s composition structure.
One of the most powerful aspects of Vari-
ant Sets is that they can vary anything about
an asset, and an asset can have multiple of
them, nested or serial. In Pixar’s pipeline, it
is common for assets to have a modeling
variant set, one or more shading variant sets,
and rigging and workflow LOD variant sets.
And More– USD also provides com-
position arcs for broadcasting edits from
one prim to many others (Inherits); special
references called Payloads for deferring the
loading of parts of a scene, to allow users to
craft manageable working sets; scene-graph
instancing to not only control the weight
of the scene graph, but be able to reliably
inform renderers and other consumers what
things in the scene can be processed iden-
tically; and many other features to facilitate
non-destructive editing of large scenes.
HYDRA IMAGING ARCHITECTURE
One of the key tools that allowed TidScene
to succeed as a project was an inspection
tool called tdsview. The tool allowed users
to inspect the contents of a TidScene
file visually in 3D and gave a very fast way
to debug shots. As we were going to
replace TidScene with USD, we needed to
provide a corresponding tool, with perhaps
loftier ambitions.
The heart of such a tool is a fast 3D
viewport, and we wanted to build as good a
viewport as we possibly could. Also by then,
some of us were beginning to itch to replace
Presto’s imaging technology. We saw a won-
derful opportunity to build a generic imaging
architecture that we could plug in to the
various proprietary tools that needed a 3D
viewport. So, Hydra was born: a multi-head-
ed beast, each head representing any one of
our scene formats and associated tools.
Hydra was originally a state-of-the-art
OpenGL-based render engine capable of
displaying feature-film scale assets in as
close to real time as we could get. Supporting
the idea that Hydra should give high-fidelity
feedback to artists, we looked to RenderMan,
our production final-frame renderer, for the
ground truth of how certain scene geome-
tries and parameters ought to be interpreted.
Along with the ability to embed Hydra into
as many applications as needed, we began
to fundamentally expand on the goals that
OpenSubdiv pioneered: high fidelity real-time
feedback that looked consistent everywhere.
Today, Hydra has grown into a much rich-
er architecture that supports not only mul-
tiple “heads” (scenes and applications), but
also multiple “tails” (renderers). We factored
out the OpenGL renderer into its own Hydra
back end that we now call Storm (though
many folks still say “Hydra” when they mean
to say “Storm” – and we’ve had to come to
terms with that, well, we’ve mostly come to
terms with that).
We’ve also implemented a RenderMan
back end for Hydra. We had already spent
an enormous effort integrating Hydra into
our various applications, like Presto, and
now almost for free, we are able to see the
scene as rendered by RenderMan. This has
the promise to be transformational for our
users, but we’d be remiss if we suggested
that we’re already able to fully benefit from
this technology. Truthfully, we’re not yet able
to take full advantage of its full potential for
several reasons.
Chief among them is that some data
required for high-fidelity rendering isn’t
available in our Presto scenes early enough
in production. Similarly, our scenes are
transformed significantly before they hit
the renderer for final-frame production. We
know we’re not done yet, and we’re excited
by the future potential workflows we’ll be
able to achieve.
PRACTITIONER TOOLSET
USD was built by practitioners, for practi-
tioners, and one of the ways in which that
e d i t i o n 2, 2020 c gw 25
8/7/20 8:13 AM
 matters is the investment in tools and
features whose goal is to make working
with USD data easier and transparent. USD
contains, out of the box, a suite of tools for
working with USD and the various external
formats that may feed USD, including:
 Binary and ASCII encodings.
 Fast, powerful preview and
debugging tools.
 Tools for creating more USD.
OPEN SOURCE, OPEN ARCHITECTURE
Pixar had experimented with open source
through its first open-source project:
OpenSubdiv. With OpenSubdiv, we wanted
to make sure that our vendors were all using
the same subdivision algorithms that Ren-
derMan was, so that artists saw consistent
and reliable results across the pipeline. We
were starting to see the value in changing
from proprietary, closed software, to em-
bracing open source as a way to strengthen
our technology and increase its longevity
by turning critical parts of our toolset into
shareable, openly available libraries that we
hoped would become industry standards.
With USD, we wanted to bring that same
consistency and reliability to the compo-
sition of the whole scene. However, unlike
OpenSubdiv, USD rests upon the lowest lev-
els of Pixar’s 3D software foundations, but
the studio strongly supported the project by
allowing us to incorporate Pixar’s proprietary
software stack into (what we hoped for
originally, and then committed to in 2014) an
open-source project.
For USD to succeed in its “universal
scene interchange” goals, it is import-
ant that 3D vendors adopt support for it
natively, and it was clear that a well-sup-
ported open-source project was the only
viable way to foster that kind of adoption.
Additionally, as we will discuss more below,
sponsoring an open-source project that
literally targets entire pipelines has been a
fantastic opportunity for productive and
enjoyable collaboration with many studios
and vendors in the industry.
Because USD needs to plug in to most
applications in any given pipeline, a critical
factor for its success is the degree to which
it can be customized, without impacting
the “common definitions” that enable
interchange. Out of the box, USD comes
with reasonable defaults for all “pluggable”
behaviors, but by writing a little code (and
26 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 26
in some cases, even without any coding),
one can:
 Teach USD how to ingest new
data sources.
 Use your own asset management
system and “asset resolver.”
 Add new schemas.
As an example of how flexible these (and
other) plug-in points make USD, Google was
able to provide Draco mesh compression for
USD entirely as a plug-in, by combining the
powers of USD references and file format
plug-ins.
With the various USD technologies
roughed in, it was time to plan their rollout
both internally to Pixar and to the broader
community.
ROLLING OUT USD INTERNALLY
AND PUBLICLY
Finding Dory was targeted as the first project
to adopt USD at Pixar. We did hit that goal,
and Finding Dory became the world’s first
feature-length film with a USD-based pipe-
line. The production also took on a significant
re-thinking of the back end of the pipeline,
taking on a pathtracing-first version of
RenderMan as well as new lighting workflows.
Given the rest of the innovations happening
on that show, USD’s rollout was relatively
uneventful, exactly as we had hoped.
In order to achieve a quiet rollout, we had
a couple of tricks up our sleeves. The first is
that we generated USD files along with our
old format, TidScene, the latter files being
the ones used for actual early production.
Our QA department also built a tool, bcsys-
tem, that would render a given shot twice
– once with USD files (not yet made visible
to production) and once with TidScene files.
The renders generated by the TidScene files
were ground truth and against which the
USD files were compared.
As long as the two renders were differ-
ent, we kept fixing bugs quietly behind the
scenes. The bcsystem tool not only tested
the renderer, but the whole pipeline that got
us to the final picture. Along with bcsystem,
a fairly big-hammer black-box tool, we also
had started amassing a big suite of perfor-
mance and regression tests. Our teams are
fiendishly fond of regression and perfor-
mance testing, keeping track of dozens
of metrics across thousands of tests. We
wanted to make sure that our new pipeline
was going to be built on software that could
produce better performance than the old.
And the best way to make sure your soft-
ware stays fast is to measure it continuously.
As we started getting renders that match
perfectly, we grew more confident that we
could switch the show over without negative-
ly impacting our users. With a flick of an en-
vironment variable, we switched Finding Dory
to start using USD and never looked back.
While we were rolling out USD on Finding
Dory, we started reaching out to trusted
partners in the industry, to whom we gave
early cuts of the software. Early partners like
Luma Pictures, DNeg, and Animal Logic, and
a little later DreamWorks, MPC, Rodeo FX,
Weta, and many others provided valuable
insight and use cases beyond our beloved
enclave that ultimately led to making USD
useful to so many in the industry.
We chose SIGGRAPH 2016 as the event
where we would officially push the “make
public” button on the USD repository
hosted on GitHub. We all gathered around a
round table set up for one of our USD Birds
of a Feather-style meetup with our small
community at the time. We pushed the
button right as folks started streaming in
for the meeting. Once everyone was seated,
we announced that USD is now public on
GitHub. It was the culmination of much hard
work for many of us on the team, and it was
overwhelming to see the support and the
enthusiasm that had already been garnered
by that time.
As of 2020, all geometry, shading net-
works and bindings, rigid-body, hair- and
cloth-simulation, and large-scale skeletal
crowd animation travel through Pixar’s pipe-
line in USD. These different kinds of data will
generally be in different files, for workflow
reasons (and there may often be multiple
shading USD files for a single asset, again for
workflow). However, being able to encode
them all as USD brings great simplifica-
tion and amplification of the power of our
toolset – not just for ease of interchange,
but also for leveraging a common language
and debugging suite between departments
and artists.
Given the advent of MaterialX and USD’s
ability to consume and encode MaterialX
networks, there may be a future where
our shaders themselves will be expressed
by standardized nodes in USD vs. custom
shading nodes driven by in-house OSL and
GLSL code.
8/7/20 8:13 AM
CGW_Issue2-2020.indb 27 8/7/20 8:13 AM
 USD IN VFX PIPELINES AND BEYOND
Today, visual effects and animation studios
throughout the world have either fully rolled
out USD or are experimenting with it for
rollout within their pipelines. At SIGGRAPH
2018, DreamWorks gave a talk titled “Zero
to USD in 80 days: transitioning feature
production to Universal Scene Description
at DreamWorks,” which highlighted how it
is possible to quickly ramp up and adapt
an existing, mature pipeline to USD. Animal
Logic, the same year, also presented a talk
titled “Forging a new animation pipeline with
USD.” There was an entire session at SIG-
GRAPH 2018 dedicated to production use of
USD, almost all outside of Pixar.
Animal Logic had taken a different ap-
proach very early on to USD interoperability
within Maya, motivated mainly by the lack
of animation support within the USD plug-in
provided by Pixar. For the first few years of
USD, integration with Maya meant choosing
between the Pixar plug-in or the Animal
Logic plug-in, or using both. Fortunately,
Autodesk, collaborating with developers at
Pixar, Animal Logic, Luma, and many other
important contributors to these projects
across several studios, is encapsulating the
two plug-ins under one umbrella with the
goal of combining the best of each into one
comprehensive plug-in. This project has
picked up momentum in the late half of
2019 and is well on its way in 2020.
The Foundry was the first to integrate
a Hydra viewport directly into its flagship
lighting and rendering software, Katana, at
first to leverage the fast preview renderer.
First-class support for USD is currently
being worked on for Katana and other
Foundry products.
In 2019, Houdini launched a new, USD-
based workflow under the Solaris banner.
Houdini is the first of the major DCCs to ship
with a deep, fundamental integration of USD
into its system.
So many other VFX and game projects
are starting to integrate USD and Hydra in
various forms, from Unity, to Unreal Engine,
to Blender, that we think that within the next
year or two, it should be fairly easy for folks
to get their hands on software that knows
how to manipulate USD.
USD has also penetrated beyond film and
games. In 2018, Pixar collaborated with Ap-
ple to design USDZ, which is a single-asset
packaged form of USD, which can contain
28 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 28
The complexity of Coco is apparent.
arbitrarily richly-composed scenes and can
be consumed just like any other USD asset,
but is suitable for Web/network transmis-
sion and archiving. Apple relies on USDZ for
all of its AR/VR applications. We have also
recently been collaborating similarly with
companies like Nvidia, which are undertak-
ing ambitious projects based on USD that
aim to propel its use in many new industries.
WHERE ARE WE HEADED?
So far, USD has been evolving at a break-
neck pace. We’re a fairly small group of
developers, getting contributions from an
industry for whom we’re grateful. While we
keep adding more and more functionality
to USD and Hydra, more plug-in points for
site customizations, and more schemas to
support more domains, occasionally we take
a step back to look at the big picture and
ask ourselves, where are we headed?
There has been a fair amount of pressure
for us to add full rigging support, and we’ve
often held the line that rigging is not within
USD’s ultimate goal of enabling interopera-
bility in pipelines. Yet, we added UsdSkel, a
capable domain of schemas for represent-
ing skeletal animation and skinning, includ-
ing blendshape support geared especially
toward representing large crowds.
Along the same lines, USD is famously
lacking support for proceduralism, a term
that is abused, misused, and confused. While
we’re not ready to commit to any particular
solution quite yet, it’s clear that USD needs to
address this issue one way or another.
How do we choose what to concentrate
on? What to work on next? The amazing
community that has grown around USD
continues to inspire us and make significant
contributions that drive the technology
forward. We’ve added features to USD that
were suggested for use cases we never
before imagined, and we hope to continue
to do so. Of course, we’re driven forward first
and foremost by the needs of the talented
artists and technical wizards here at Pixar,
without whom we wouldn’t have anything
worth writing about and for whom we are
eternally grateful.
Speaking of being eternally grateful,
we’d like to take this opportunity to thank
our amazing team: Sunya Boonyatera, Alex
Mohr, Corey Revilla, Varun Talwar, Pol Jere-
mias-Vila, David Yu, Kees Rijnen, Matthias
Goerner, Raja Bala, Tom Cauchois, Florian
Zitzelsberger, Adam Fish, Adam Woodbury,
Florian Sauer, John Loy, Brett Levin, Matt
Kuruc, Stephen Gustafson, Jilliene Tongson
Paras, and Jessica Tran. We’d also like to
thank our Infrastructure team led by Cory
Omand, and our Applications team led by
Hayley Iben, and all of Pixar’s engineers and
artists. We owe USD to the trust that our
VP, Guido Quaroni, put in us and to our CTO,
Steve May, and the rest of the Executive
Team at Pixar who helped create the
wonderful environment that allowed USD
to flourish. Finally, to all of our contributors
(look for a list here http://graphics.pixar.
com/usd/docs/USD-Contributors.html), a
huge big thank you. 
F. Sebastian Grassia and George ElKoura are lead
software engineers at Pixar Animation Studios.
8/7/20 8:13 AM
 next issue
FALL ISSUE (Sept., Oct., Nov.)

¢ Animation: Pixar’s latest animated film shows it has Soul.

¢ Gaming: A look at the new gaming technology.

¢ Stylized Animation: Glen Keane’s Over the Moon takes audiences on a moving journey.

¢ VR: We pull back the curtain on some of the latest VR technology.

FOR ADVERTISING INFORMATION CONTACT

MARI KOHN, DIR. OF SALES | 818.291.1153 | mkohn@copcomm.com
LISA NEELY, MARKETING & ADVERTISING SALES EXEC. 818.660.5828 | lneely@copcomm.com
WILLIAM R. RITTWAGE, PRESIDENT/CEO 818.291.1111 | brittwage@copcomm.com

News happens quickly in

our industry. Keep informed.
Keep up to date.
Anytime. Anywhere.

SIGN UP TODAY!

WWW.CGW.COM

CGW_Issue2-2020.indb 29 8/7/20 8:13 AM

 A Bedtime Story
FILMMAKERS USE VR TO EXTEND THE WORLD OF THE ELEMENTAL SPIRITS
IN ‘MYTH: A FROZEN TALE’
BY KAREN MOLTENBREY
B
efore children close their eyes and
drift off to dreamland, parents will
regale them with bedtime stories
of fantastical characters from wondrous
worlds. Disney’s third VR film, “Myth: A
Frozen Tale,” uses this ritual to kick off a fairy
tale like no other, taking viewers on a visceral
journey and immersing them in a stylized
world inspired by the environments and
elemental characters from Frozen 2.
“Myth” begins in a forest setting outside
of Arendelle, as a local family sits down to
hear a bedtime story. As the mother reads,
the audience is transported to a mystical
and stylized world inspired by the elemental
spirits, themes, and environments introduced
in Frozen 2 come to life, and the myths of the
characters’ past and future are revealed.
The VR short was created in-house at
Walt Disney Animation Studios by the
core team who built Disney’s first VR
film, “Cycles.” “Myth” was directed by Jeff
Gipson, who also directed “Cycles” (2018).
Jose Luis Gomez Diaz, who also worked on
“Cycles,” served as VR technology super-
visor on “Myth,” with Michael Anderson as
VR environment lead and Ed Robbins as
VR character lead. Brittney Lee, a key visual
development artist on Frozen and Frozen
30 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 30
2, was the production designer. In fact, the
same artists, technologists, and teams that
work on the studio’s feature films work on all
the VR/AR projects, too.
The impetus for the project began with
Jennifer Lee, Disney Animation’s chief
creative officer, who wrote and directed the
Frozen films, after approaching Gipson in
late 2018 about exploring the Frozen world
for his next VR project.
“’Myth’ is a fairy tale, a bedtime story in a
way of a myth or legend about the ele-
mental spirits from Frozen 2,” says Gipson,
who was inspired by bedtime stories from
his past. He tried to imagine what type of
stories he would have been told as a child if
he were to have grown up in Arendelle. Then
after watching those early screenings of Fro-
zen 2, he became drawn to those elemental
characters, and pondered what they would
like look in VR.
“I like the idea that if we grew up in Aren-
delle, this was something that everyone was
told, a shared story,” he adds.
A Unique Style for a Unique Medium
In Frozen 2, those elemental characters
assumed elaborate 3D forms. However,
Gipson’s vision for them in his VR storybook
world was inspired by pop-up books and
graphic silhouettes, even stage elements
from vaudeville and music hall productions.
That style meshed well with Lee’s aesthet-
ic, which Gipson describes as graphic and
stylized, almost Eyvind Earle-like, referring to
the art director on the 1959 animated Sleep-
ing Beauty. “She does a lot of cutout paper
art as well. That was one of the reasons I
gravitated to her artwork and her style,” says
Gipson of Lee.
As Lee points out, the world of Frozen 2 is
already stylized, and with “Myth,” they styl-
ized it further. “We wanted to be respectful
of what was already there, and didn’t want
to take the design out of its original realm.
It was challenging to walk a fine line that
elevated it and took it in a new direction
but still felt like it would fit into the Frozen
spectrum,” she adds.
Inspiration for the VR short’s music-driv-
en narrative came from Disney’s Fantasia
and even Peter and the Wolf, with all the
characters having their own piece of score. “I
wanted music to play into that, where every
character has protagonistic characteristics,
but also antagonistic characteristics as well.
The music can inform each one of those
and then balance itself out as we introduce
8/7/20 8:13 AM
 The monolith at day (top) and at night (bottom).
fire, air, earth, and water,” says Gipson.
“It’s almost like a visual poem in a way. We
harnessed 2D traditional animation from the
artists here in the studio to be incorporated
into this new medium.”
The stylized elemental characters in
“Myth” project a duality that intertwines well
with the music. The Nokk, a mythical water
spirit that takes the form of a horse that
harnesses the power of the ocean; the Nokk
fiercely guards the secrets of the forest and
those wanting to pass must earn its respect.
Gale, the playful wind spirit whose presence
can take the form of a soft breeze or a raging
tornado. The massive Earth Giants, spirits of
the earth that take the form of rocky river-
banks while sleeping, but when awake and on
the move, are capable of massive destruc-
tion. And, the fire spirit, a tiny fast-moving fire
salamander named Bruni, whose emotions
affect the intensity of its power.
“I began thinking of these elemental
characters as members of a band, and when
they’re all playing together, they help to keep
the world in harmony,” says Gipson. “When
even one of these spirits gets out of sync,
the harmony is broken.”
CGW_Issue2-2020.indb 31
Like in Fantasia, each character has its
own score, color palette, and language. “I
wanted to bring that element of Disney
heritage into the medium of VR. Just as
there is a duality to each of the elemental
spirits, there is a duality to the music, which
runs the gamut from sweet to antagonistic,”
Gipson points out. The film’s original score,
which drives the storytelling experience,
was composed by Joseph Trapanese (Tron:
Legacy, Raid series).
“Myth” producer Nicholas Russell, who
had worked with Gipson on “Cycles,”
acknowledges the particular inspiration
Fantasia had on the short. “We wanted to
combine that type of filmmaking with our
newest technological tools. We kept imag-
ining what it would be like if you could stand
next to Mickey Mouse as the Sorcerer’s
Apprentice when he’s conducting the stars
or battling the brooms,” says Russell.
Myth-ical Characters & Myth-ical Lands
For planning the scenes, the artists used
Oculus’ Quill, a VR painting tool, to sketch
out scenes for determining scale, the prox-
imity of characters, and so forth.
Of course, foremost, the stylized charac-
ters and world of “Myth” had to work within
virtual reality, which meant fleshing out the
whole world and showing the viewer what
is happening behind them, above them, and
over there in the distance, all while telling the
story. And that was no easy task consider-
ing how rich the short film is with effects,
which are essential to the narrative and the
experience, Anderson notes. “It’s a give-and-
take. All that has a cost, and it’s a really big
challenge to balance all of the ideas and the
cool work coming in, with the performance
that we had to hit to make this viable,” he
says. “It was a constant battle.”
The group was intent on obtaining the
same or close to the same quality as the
feature film, and still maintain the necessary
frame rate. “Real time is running at way fast-
er speed than what we render in the studio,
so there are limitations,” Robbins empha-
sizes. As a result, the team had to be very
clever on how it preserved that quality.
“For the most part, we preserved as much
of the pipeline that made sense, and then
whenever it didn’t make sense, we filled that
void with what we needed,” Anderson says.
e d i t i o n 2, 2020 c gw 31
8/7/20 8:13 AM

The fire spirit Bruni, a quick-moving salamander, heats up the action.
For the characters, the team reused as
much of the feature film’s production pipe-
line as possible, beginning with the same
assets and animation and simulation rigs,
up until the point of animation – at which
time they branched off into a customized
setup. Also, they used Swoop, the in-house
tool developed to give artists the ability to
animate the path and timing of the wind
spirit Gale in Frozen 2, either directly within
Autodesk’s Maya or by using VR. Due to its
VR capabilities, Swoop enabled the “Myth”
team to puppeteer objects and characters,
creating the shape of the motion paths and
the timing for the characters to travel over
that path.
For the environments, all the models
likewise had to be reprocessed so they
could be handled within Epic Games’
Unreal Engine, the VR platform used by the
filmmakers for “Myth.”
“We take lower-res models and have to
create UVs, then we have to create fully
unique textures. None of that really transfers
from our regular pipeline. It’s a completely
different system,” says Anderson. “So we
make new textures for our models using
[Adobe’s] Substance Designer and various
other tools.”
Once the models were brought into the
Unreal Engine, shaders were created and
set up so they could be rendered in the
engine. While typically animation is exported
through Disney’s proprietary format, in this
case it was exported as cached FBX files
and then brought into Unreal Engine, where
the layout was done inside the Unreal En-
gine Sequencer Editor tool.
It’s important to note that the stylized
characters and world of “Myth” provided the
artists with more freedom in making creative
choices when it came to their assets. In the
VR short, the Nokk, an extremely complex
character, is featured prominently. And while
the team started off by using the Nokk’s body
asset from the feature film, they decided to
make Nokk’s mane a cloth simulation sheet
32 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 32
as opposed to a fluid simulation, which was
used in the feature and is far more complex
and elaborate, notes Robbins.
From ‘Cycles’ to ‘Myth’
As Gipson explains, there are many ways
to make a VR film, though he prefers the
narrative, linear approach of “Myth” and his
previous “Cycles,” as opposed to a highly in-
teractive experience where the user is con-
stantly grabbing at items with controllers.
There is a versatility to the way you can
tell stories in VR, and the way that audienc-
es can interact with the world, Lee points
out. “It’s very different from what we do in
feature films, and I think there’s something
very fun and exciting about it. You can
really get into that space, and viewers can
experience the elements a bit more closely
and on a personal level,” she says. “I feel like
that may be the most enticing aspect of it.
There’s a real ‘wow factor’ when you get to
have facetime with the Nokk and he comes
right up to you and assesses whether or not
you’re worthy. You can almost feel his breath
on you as he approaches. And the scale of
the Earth Giants is so immersive to the point
where you feel that they can practically pick
you up. Experiencing the Northern Lights
as if you’re standing right in front of them is
also truly magical.”
In “Cycles,” the filmmakers tried to guide
the eye of the viewer using color and light,
as well as motion, to push the viewer around
the space. They also employed a technique
whereby the space darkens and desaturates
somewhat if the person is not looking in the
intended area. When it came to “Myth,” they
again used color, light, and motion to the
same effect to direct the viewer’s attention.
Also, the interactivity is gaze-based. Users
have six degrees of freedom, whereby they
can move and view the film as they like. But
there are also moments where the char-
acters will react based on the user’s viewer
direction or proximity to the characters.
Indeed, “Cycles” (which was created
with Unity instead of Unreal Engine) was a
learning experience, being the first project
of its kind for the Disney Animation group.
“When we opted to go with a different
engine for this film, that meant starting
from scratch in a lot of ways,” says Ander-
son. “The pipeline changed because the
platform changed. We were able to do a lot
more and iterate faster because of the tool
set we were using [for ‘Myth’].”
There were other differences in the
processes, as well. For the characters in
“Cycles,” the artists used Alembic, a cached
data format. For “Myth,” they used cached
FBX rigs. Also, “Cycles” has a more tradi-
tional look, as artists used a toon shader to
flatten out the characters. As for the envi-
ronment, it is a single setting that remains
fairly consistent. Conversely, in “Myth,” the
environments are diverse, and artists visually
leaned into the art direction and shape
language familiar to Frozen. Colors change to
suit the story, with no cuts.
“It has to be driven in one continuous
piece,” says Anderson. “It’s a technical chal-
lenge to have the sky change and the colors
change, and cut from the cabin into the
forest in a creative way so it’s not just a fade
to black. All of these things are simple and el-
egant when you watch them, but underlying,
the technology is very complicated, and it’s
complicated to get it all working in harmony.”
“It’s amazing to see [Gipson’s] range in
going from a very personal and emotion-
al film that was seen in the confines of a
house like ‘Cycles,’ to this latest project,
which encompasses an entire forest and
the whole of nature,” says Russell. “This film
is a game-changer in the way that the VR
technology is being used to tell a story and
make the viewer a part of that world.”
Closing the Virtual Chapter
“Myth,” which is approximately eight min-
utes in length, took close to eight months to
complete following creative approval, with
production wrapping last November. All told,
about 80 people worked on the short film,
though a core team of eight or so worked on
the film for its entirety.
“Myth: A Frozen Tale” is now available on
Facebook’s Oculus Quest.
… The end. 
Karen Moltenbrey is the chief editor of CGW.
8/7/20 8:13 AM
 OVERALL CG MARKET TO GROW TO $154B BY 2023
BY JON PEDDIE
The computer graphics market has always
been an exciting place since it began in the
early 1960s. Challenged by hardware speed
and costs, visionaries saw the potential, and
driven by artistic and scientific quests, pushed
the envelope seeking imagery and moving pic-
tures that were indistinguishable from reality,
as well as fantastic and imaginary.
Computer graphics is not a single dis-
cipline, but rather a collection of capabili-
ties and people: processors, engineers, lots
of memory, artists, displays and XR, tech-
nical directors, programmers, and zillions of
lines of code.
So, the universe of CG has continuously
grown, exponentially at times. Sometimes
it has followed Moore’s law, sometimes it’s
been boosted by software breakthroughs
and eureka moments. Growth in the dollar
value of the market has leveled off a bit, but
that’s due as much to falling prices as it is
to the asymptotic nature of any market. As
a matter of fact, CG has seen growth for
over 60 years – not many markets can make
such a claim.
Growth has slowed to an average (hard-
ware and software) of 1.2% CAGR, but at
$154 billion a year, that’s not trivial.
The software aspects of the market have
been slow to flat, but hardware segments
have been varied, influenced by consumer
products such as game consoles and PC
gaming machines.
Virtual reality, which has gained so much
attention, is hardly a factor in either hard-
ware or software, and despite some glowing
forecasts, we don’t think it will be a factor
COMPUTER GRAPHICS HARDWARE MARKET
Monitors used for graphics
Workstations
Game consoles
Total gaming PCs
Mobile devices
TOTAL hardware ($B)
CGW_Issue2-2020.indb 33
for a while. A growing point of exploration
for VR today is in content creation where it
is used for virtual production, a glamorous
but tiny market. The real market for VR is in
the design fields for professionals.  spaceships. The beautiful ray-traced images
The computer graphics hardware market
was worth $130 billion in 2018 and is expect-
ed to increase to $137 billion by 2023, with
software growing slightly and a little less
than hardware.
The hardware segment of the CG industry
has seen steady overall growth, with some
ups and downs due to product cycles and
the irregularity of the PC market.
The year 2019 witnessed the intro-
duction of new GPUs and add-in boards
(AIBs) with ray-tracing capabilities; several
new CPUs and next-gen game consoles
also were announced for the second half
of 2020.
The market for CG software in 2018 was
worth $14 billion (not counting services,
maintenance, and other aspects) and is
expected to grow to $17 billion by 2023 as
the industry shifts its sales model, moving
more services to the cloud and transitioning
to a subscriber system.
Although it doesn’t get discussed much,
the digital graphics market is dominated by
CAD/CAM. Digital video is a bigger market
than the modeling or rendering segments,
and yet when one reads about CG, it’s all
about the special effects in games and
movies. That’s easy to understand. Everyone
loves VFX and has experienced it. In reality,
truly amazing things are happening in CAD
as the digital and real get connected. CAD
2018 2019 2020
1.00 1.00 1.08
10.48 12.37 13.08
14.81 12.06 12.47
35.48 34.57 35.91
68.63 75.35 70.73
130.41 135.35 133.27
has taken a leap, and it shows in increased
revenues for all companies. And, although
unsung, CAD is often involved in the heavy
lifting of all those 3D models of cities and
of automobiles, gorgeous skyscrapers, and
clothing, bottles, and other consumer goods
– it’s all CG.
The demand for programmers, artists,
scientists, and designers continues to be
strong, and we’re seeing start-ups arrive in
emerging and reborn markets such as aug-
mented reality, virtual reality, and casual
games. Artificial intelligence is becoming
increasingly important in image process-
ing, ray tracing, and AR. The arrival of new
APIs and platforms are also stimulating
development. Firms are actively looking
for people who can use and exploit these
new programs and their associated hard-
ware accelerators. 
Given the trends in dropping costs, and
the increasing users and usage of CG tools
and hardware, we predict that the rate of
growth for the CG industry will remain fairly
steady for the foreseeable future.
Computer graphics is truly a worldwide
industry now.  
Dr. Jon Peddie (jon@jonpeddie.com) is a
recognized author and pioneer in the graphics
industry; president of Jon Peddie Research, a
Tiburon-CA-based consultancy specializing in
graphics and multimedia that also publishes
JPR’s “TechWatch”; and named one of the most
influential analysts in the world.
2021 2022 2023
1.18 1.27 1.32
14.50 15.90 17.16
17.29 17.25 16.45
35.64 37.00 36.95
65.70 66.70 65.37
134.31 138.12 137.25
e d i t i o n 2, 2020 c gw 33
8/7/20 8:13 AM
 Medical animators Alissa Eckert and Dan Higgins of the CDC created this image of the COVID-19 virus.
Left Brain, Right Brain
ANIMATORS USE COMPUTER GRAPHICS TO HELP EXPLAIN COMPLEX MEDICAL CONCEPTS
BY KAREN MOLTENBREY
I
n the early part of the year, COVID-19
struck, and struck hard and fast. In
a matter of a few months, this novel
coronavirus infected close to 4.7 million
people in the US alone, killing 160,000 of
them as of early August, and those number
continue to rise with each passing day.
Indeed, this microscopic monster has
impacted the lives of people around the
globe in ways we could never have imagined.
And yet, it’s safe to say that most of us have
never seen this virus, although all of us can
instantly recognize it on sight, thanks to the
work of medical illustrators and animators.
These specialized artists use illustrations
and animations to help the medical com-
munity as well as scholars and the public ac-
curately communicate complex information,
and in the case of COVID-19, put a “face” to
this invisible infectious agent.
The work requires significant education
and training, both in the medical/science
fields and in the arts – a left brain/right brain
34 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 34
profession requiring a very specific type of
skill. Here we look at three medical illustra-
tors and how they are using the same type
of 3D tools and techniques employed by
filmmakers and others to ply their craft.
Alissa Eckert and
Dan Higgins, CDC
Alissa Eckert and Dan Higgins are both
medical illustrators at the CDC in Atlanta,
and it was their work that led to the image of
COVID-19 that is seen everywhere, from the
covers of magazines, to charts and graphs
used by medical experts, to the nightly news
broadcasts.
Eckert initially planned to become a veter-
inarian, but has always taken art classes for
fun. She started out studying biology for a
few years and then switched to scientific
illustration, an interdisciplinary study that
combines science and art, after discovering
that she could combine both those inter-
ests into one profession through medical
illustration. She looked into it as a career
and determined it was for her. She eventu-
ally earned a master of science degree in
medical illustration, having had to com-
plete classes alongside medical students,
in addition to art classes that focused on
illustrating scientific information. Right out
of school, she began working at the Centers
for Disease Control (CDC), where she has
been employed for 14 years now.
8/7/20 8:13 AM
 For her colleague, Dan Higgins, his interest
in art guided him into this career. He was
working as a graphic designer until an
interest in science led him down the road to
a master’s degree in medical illustration. He
began his new career as a medical illustrator
with WebMD before he migrated to the CDC,
where he has been employed for the past
18 years.
“We’re kind of like translators for the
scientists when they’re communicating with
the public,” says Eckert. “We take complex
scientific information and turn it into visuals
that people can easily understand.” This
could pertain to a wide range of topics at
the CDC – from viruses and bacteria, to
tuberculosis, AIDS, cancer, chronic health
conditions, birth defects, and so much more.
While the work can be geared toward
the general public, the two artists also
do graphics, illustrations, and animations
that are intended for the scientific com-
munity, including physicians and scientists
on staff at the CDC for lectures, journal
articles, and presentations, where they
are conveying the information to fellow
scientists and researchers.
Just as the subject matter is diverse,
so, too, are the image formats used: 3D,
3D with animation, 2D and 2D animations,
illustrations, 3D printing, and more. Eckert
and Higgins use Autodesk’s 3ds Max as their
primary 3D software, along with Pixologic’s
ZBrush. They also utilize Adobe’s Creative
Suite as well as a number of plug-ins,
including those from Red Giant (which now
merged with Maxon) and Video Copilot.
The artists additionally employ special-
ized software called Chimera, an extensible
modeling system for the interactive visual-
ization and analysis of molecular structures
and related data, including density maps,
trajectories, and sequence alignments,
from which high-quality images and movies
can be made. Chimera is developed by the
Resource for Biocomputing, Visualization,
and Informatics (RBVI) at the University of
California, San Francisco.  of back and forth with the scientists. “We
According to Eckert, she uses 3D
as much as possible, which eases the
workflow. “I find the process much faster,
especially when small changes have to be
made,” she notes, whereas in the past when
they were producing more hand-drawn il-
lustrations, changes were time-consuming.
In addition 3D, these medical illustrators
CGW_Issue2-2020.indb 35
At the CDC, medical animators use a range of DCC software for their work.
have been utilizing 3D printing in their work,
output on a 3D Systems’ ProJet 460Plus or
a FormLabs printer.
Eckert recalls making a 3D printed model
a year ago that was used to train physicians
in countries where a specific birth defect
was prevalent, to teach them how to take
accurate measurements. “It’s a life-size
model, so they can practice measuring tech-
niques, which involves a very specific pro-
cess,” she explains. Other 3D printed models
of viruses and so forth have been used for
informational and educational purposes,
sometimes for a conference or lecture.
“The tactile experience [from a 3D-print-
ed model] really helps people learn about
the structures,” Eckert adds.
When the work involves an animation, the
artists will consult with scientists through-
out the project, making sure the science is
accurately depicted.
Some projects take quite some time to
complete. But, when COVID-19 struck so
quickly, Eckert and Higgins had just one
week to come up with an image for the virus.
“We had to drop everything we were work-
ing on to focus on this one project,” says
Higgins, noting that in a typical scenario, the
work would have taken at least three weeks
were granted access to the scientists much
faster, and they were able to guide us on
which proteins we needed to include and
the quantity of the protein.”
In this instance, it was important to grab
the viewer’s attention. “This was a public
health emergency, and we needed them to
pay attention to this topic and hopefully go
to the CDC’s website to learn more informa-
tion,” explains Eckert.
In addition, the two medical illustrators
were putting a face to this virus, taking
something that is unfamiliar and invisible,
and bringing it to life so people would realize
that it is real and that it does exist. “We
wanted people to take this seriously, so we
created it in a certain way,” adds Higgins. To
this end, the artists created a gray sphere
and then added bold red S-proteins –
spiked proteins (which attach to a cell)
that are a signature of the coronavirus and
enable it to be identified and recognized.
“You would see a similar shape when
you look at it on the electron micrograph,
though they [the virus] are always kind of
fuzzy and hard to read. We’re just making
it more clear, because they are really, really
tiny,” Eckert says.
The main goal in this case was commu-
nication, and as such, the artists had to
balance the colors so that the image was
bold, vibrant, attractive, and noticeable
without being too playful, so it would be
taken seriously without scaring people. “We
wanted people to look at the image and
become interested in it,” says Eckert, noting
she and Higgins created a number of vari-
ations of the image before settling on this
very popular graphic.
“We were intending it to be an education-
al opportunity, and it just blew up. I never
thought it was going to do that,” says Eckert
of the image’s widespread popularity.
Higgins agrees, adding, “We’ve seen our
work out there before, but this just exploded.
I even saw it on SNL, which was pretty crazy.”
e d i t i o n 2, 2020 c gw 35
8/7/20 8:13 AM
 Veronica Falconieri Hays,
Falconieri Visuals
Veronica Falconieri Hays is a certified med-
ical illustrator specializing in medical, mo-
lecular, cellular, and biological visualization,
encompassing still media and animation, at
her company, Falconieri Visuals, based in the
Washington, DC, area.
Falconieri Hays received her undergrad-
uate degree in biology from Smith College,
with a minor in art, before earning a master
of art degree in medical illustration, “which
covered a lot of the basics of how to create
accurate illustrations and the general
techniques that are used in a lot of the
medical and scientific visualizations today,”
she says.
After graduating in 2014, Falconieri Hays
worked at the National Cancer Institute as
an illustrator and animator before launching
her own company in 2017.
In order to become certified in this
field, Falconieri Hays has to meet con-
tinuing education requirements and pass
an exam by illustrating art, medical, and
business knowledge.
“Everything I do translates scientific
information, research, understanding, and
concepts into visuals. This means doing
a lot of research and having a fair amount
of literacy in the underlying concepts, and
then adapting that visually to the appro-
priate audience, whether to experts or the
public,” she explains. “It’s about making that
connection between complex ideas and
translating it visually to the right audience.”
Naturally, Falconieri Hays was always in-
terested in both science and art, and heard
about this field while she was in high school.
“When I was pondering my future career
and college choices, I initially thought that I
either had to choose science or art. Fortu-
nately, I heard about medical illustration and
realized it would enable me to incorporate
both into a career,” she notes.
Falconieri Hays’ client list comprises
communications agencies that work with
biotech companies which have complex
science they need to explain to investors, for
instance. Others are researchers and scien-
tists themselves who need help with visuals
to explain the work on a research project.
She also produces imagery for publications,
such as Scientific American (a recent issue
cover image and main article image contains
an illustration the artist did of SARS-CoV-2,
36 cgw e d i t i o n 2, 2020
the COVID-19 virus, in 3D).
Depending on the audience, the imag-
ery can be scientifically precise, or such
that Falconieri Hays can play with scale or
use metaphor. It just has to be clear and
understandable to the audience for which it
is intended.
“I focus on molecular and cellular visu-
alization, using 3D,” Falconieri Hays says. In
particular, she employs Maxon’s Cinema
4D – which she learned in grad school –
because “it has a plug-in, ePMV, that allows
me to directly pull molecular data. I also use
[Pixologic’s] ZBrush for organic modeling.”
Once the modeling and animation is
completed, Falconieri Hays uses various
Adobe tools “to bring it together.” Render-
ing is done either with the Cinema 4D’s
standard renderer or with the Redshift
renderer, depending on the look she is
going for. For compositing, she uses Adobe
After Effects.
When creating her work, Falconieri Hays
will spend a great deal of time collaborating
with scientists and researchers, who will
identify the scientific concept and story they
want to convey, as well as their audience for
the project. They will also identify the format
they need, whether it’s video or still images.
Then she begins sketching with pencil and
paper. Once the client agrees that all the
important points are covered and correct,
she begins the creation process, whether it’s
in 3D or 2D.
As the recent pandemic was developing,
Falconieri Hays, along with colleagues at the
Association of Medical Illustrators, created a
shared reference document, which included
an illustration summarizing the 3D structural
features of the SARS-CoV-2 Spike (S) pro-
tein structure (SARS-CoV-2 is the virus that
causes COVID-19). “Research was coming
out so fast; it’s a lot to keep track of. So, we
decided to collaborate and help one another
by crowd-sourcing a lot of our reference to
help medical illustrators as a whole,” she
explains. “And because I have a particular
expertise in the 3D molecular structure, I
wanted to create something to give back to
my colleagues if they were asked to visualize
the spike protein, for example. This way,
they’d have a better understanding of how it
works because it’s pretty complicated.”
For this particular illustration, Falconieri
Hays looked closely at the 3D structure of
the spike proteins, which are the hook-like
Medical illustrator Veronica
Falconieri Hays created this image
of the SARS-CoV-2 virion, the
COVID-19 virus, in 3D.
pieces that project out from the surface of
SARS-CoV-2. This work required a great deal
of research. In fact, out of the approximately
27 hours it took for Falconieri Hays to do
the project, at least half of that was spent
conducting research, “just making sure I
understood the basis of the virus structure
and the unique details about the spike
structure, because it’s a target for vaccine
development,” she explains. “It’s an area
that an antibody, for instance, could bind
to and block the virus from interacting with
human cells.”
According to Falconieri Hays, animations
are becoming a larger part of the work she is
asked to do, often to show how a therapeu-
tic works.
While Falconieri Hays may not be creating
visuals for the next blockbuster, the work
she does is extremely important. “I do a lot
of illustrations that are scientifically com-
municating to other scientists, but I also do
projects that are more public-based,” she
says. “It’s really important having amazing
visuals because the subject matter is so
complicated. You’re asking a lot of your
viewer to step in and really take the time to
understand the science you’re communicat-
ing. If you attach an enticing visual to it, you
get a little more buy-in. You can communi-
cate more science to the viewer if you have
a visual that hooks them and piques their
interest.” 
Karen Moltenbrey is the chief editor of CGW.
 S2020.SIGGRAPH.ORG
SIGGRAPH 2020 offers inspiration, putting
the latest in art and tech at your fingertips.
Discover inspiring content that demonstrates
the latest advancements in computer
graphics and interactive techniques research,
education, applications and entertainment.

The 47th International Conference & Exhibition

on Computer Graphics and Interactive Techniques

CGW_Issue2-2020.indb 37 8/7/20 8:14 AM

 Working Together
FOR M&E WORKSTATION PERFORMANCE, IT’S DIFFERENT STROKES FOR DIFFERENT FOLKS
BY BOB CRAMBLITT
W
hen one talks about workstation
performance related to product
development, CAD/CAM/CAE
usually comes to mind. It’s fairly easy to link
design time-savings to greater productivity
within the product development cycle. Yet,
professionals producing graphics for media
and entertainment (M&E) applications such
as gaming, special effects, and 3D animation
face very similar challenges when seeking to
optimize their workflows.
Just as one type of workstation config-
uration does not fill the needs for all CAD
applications and workflows, neither does it
for media and entertainment. Performance
is based on a complex web of interactions
associated with the application itself, the
system configuration, the type of models
being constructed, rendering complexities,
the myriad paths data and storage take, and
the destination for the end product.
What’s good for Autodesk 3ds Max per-
formance might not be as good for Maya.
Likewise, what works for gaming might not
work as well for special effects, 3D anima-
tion, or architectural simulations.
NO TWO MODELERS ARE THE SAME
Modeling in M&E applications is marked by
a series of interactions as the artist makes
changes to the model and those changes
are visualized in the viewport. The modeling
Demanding apps such as Blender benefit from massive multi-threading.
38 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 38
process might appear simpler than it is in
CAD, but looks can be deceiving, according
to Alex Shows, chair of the SPEC Worksta-
tion Performance Characterization (SPEC­
wpc) subcommittee. “One of the differenc-
es in modeling demands between CAD and
M&E is the diversity of geometry versus ren-
dering complexity,” says Shows. “CAD spans
a wider gamut of geometric complexity,
while M&E applications incorporate some of
the most sophisticated interactions among
an incredibly diverse palette of materials.”
M&E models also have fewer demands
placed on them within the design pipeline,
according to Ross Cunniff, chair of the SPEC
Graphics Performance Characterization
(SPECgpc) subcommittee. “CAD modeling
is typically driven by requirements of the
back-end CAM pipeline, leading to design
decisions within the application that can
sometimes reduce graphics performance,”
says Cunniff. “In M&E applications, the
image produced is the product.
Unlike CAD, most models for M&E need
to be only marginally functional.
“M&E models typically just have to look
good or be reasonably functional,” says
Trey Morton, chair of the SPEC Application
Performance Characterization (SPECapc)
subcommittee. “The time frame for M&E
objects on the screen can be extremely
short and the objects often do not require
great detail. A landing gear for a spaceship in
a video game, for example, just needs to dis-
appear into the fuselage, where on an actual
CAD model of a space shuttle it needs to
properly fold up and be stowed.”
Both CAD and M&E modeling are single-­
threaded operations that improve with CPU
frequency. But as Shows points out, M&E
material complexities place a much greater
strain on the GPU and VRAM to deliver the
frames per second needed for refresh-
ing graphics on the screen when making
changes to the model and interactively
rotating, panning, and zooming around the
scene.
Modeling functionality is not monolithic
across different M&E packages, of course.
Beyond the differences in end-product
requirements – size of the models, complex-
ity of textures and effects, resolution of the
final images, and length of time displayed on
the screen – there are inherent differences
in how the applications themselves handle
modeling. There are also the continuous
changes within versions of the same appli-
cations as new technologies are implement-
ed to speed operations and increase realism.
“Different implementations between
different versions of an application can make
a large difference in the hardware required to
maximize performance,” says Cunniff. “Older
versions of [Autodesk’s] Maya, for example,
required significant CPU overhead to transfer
work to the GPU. Newer versions can store
work directly on the GPU, removing the bot-
tleneck and making higher-end GPUs much
more useful for high-performance modeling.”
SAVING RENDERING TIME
TO REINVEST
For both interactive and non-interactive
rendering in M&E applications, the focus
shifts from single threads to multi-threading,
where more cores working in parallel can
significantly accelerate performance. This is
especially the case when dealing with very
8/7/20 8:14 AM

Heavy use of textures, like in this model, puts a strain on GPUs and VRAM.
large, realistic models produced in high-res-
olution for television and movie production,
and architectural models incorporating real-­
world shadows, reflections, and textures.
“The critical components for increasing
rendering performance in an M&E applica-
tion are high core count processors, whether
they are general-purpose CPUs or massively
wide vector processing engines like the
compute cores found in a GPU,” says Shows.
“Which of these is more important depends
on your application and workflow.”
Interactive rendering, when an artist makes
a change to a scene and wants to quickly see
the newly applied textures and lighting in the
viewport, is heavily reliant on the massive
computing cores found in the GPU.
The speed of both interactive and non-in-
teractive rendering strike at the core of pro-
ductivity: A fast GPU with enough memory
will calculate changes on the fly, eliminating
waiting time for the artist. A powerful GPU
can also cut down overall render times for a
scene and save a studio hours of time that
can be reinvested into the artistic process or
later production phases.
PARALLEL PROCESSING
SPEEDS ANIMATION
The animation process – creating or
changing motion for characters and objects,
then seeing the scene played back in real
time – is heavily reliant on multi-core CPUs
and GPUs. Maya is particularly optimized for
animation operations, incorporating parallel
processing and playback caching to stream-
line performance.
Parallel processing assigns a computa-
tional task to the best available core, while
cached playback saves calculations to
system or GPU memory so the application
doesn’t need to load the changes to every
frame, enabling smooth playback. “Animators
looking for the best performance need to
CGW_Issue2-2020.indb 39
invest in a high-core-count CPU and a high-
end GPU with enough memory to cache full
animation sequences,” says Cunniff.
WHAT’S THE FINAL PRODUCT?
As with CAD, the end result of the M&E
production cycle has an impact on what
configurations will provide the best ROI
for hardware investments. Film and TV
sequences are rendered at 4K or sometimes
8K resolution per frame at 24 or 60 frames
per second. The large number of high-reso-
lution, uncompressed frames places a pre-
mium on network or local storage to address
the high data rate demands.
Game development has dual require-
ments: creating the game in applications
such as Maya, 3ds Max, or Blender, then
integrating the assets into game engines
such as Unreal or Unity. The emphasis in
this process is optimizing animation and
effects such as lighting and shadows to
create a seamless gaming experience. A
modern GPU with multiple cores and a large
VRAM buffer gives game developers the
speed they need to create scenes and make
changes interactively. A fast CPU then allows
quick transfer of files into the game engine.
Other functionality within M&E appli-
cations can stress computing resources
as well. Transcoding, the act of converting
a media file or object from one format to
another within an application, puts a tax on
a workstation and can benefit from GPU
acceleration, according to Cunniff.
MEASURE TO GET BETTER
Beyond ascertaining how different M&E
applications interact with workstation com-
ponents, professionals concerned about
performance should be aware of their own
usage patterns.
Ideally, workstation users would devel-
op their own benchmarks specific to their
workloads, the M&E applications they
use, and the functionality they rely on the
most. But, this is difficult, and the terrain is
constantly shifting, as new versions of appli-
cations introduce new ways of maximizing
efficiency.
The next best thing to homegrown perfor-
mance measurement is to use standardized
benchmarks that most closely align with
your application and provide the granularity
to measure the myriad paths that an appli-
cation takes. In that vein, there are SPECapc
benchmarks based on 3ds Max and Maya
that run on top of those applications. If you
want a smaller, self-contained benchmark,
the current version of SPECworkstation
includes workloads for 3ds Max, Maya,
Blender, and open-source applications such
as the Handbrake media encoder and the
LuxCore physically-based renderer.
If you are interested primarily in graphics
performance, the SPECviewperf benchmark
(available in Windows and Linux versions)
provides workloads (called viewsets) based
on 3ds Max, Maya, and Showcase. All of
these benchmarks are available to users for
free download. Vendors of computer-related
products and services that are not mem-
bers of SPEC/GWPG are required to pay a
licensing fee.
THE EVOLUTION NEVER QUITS
Users don’t need the biggest or fastest of
every component, but will benefit from
upgrades that address their most problem-
atic bottlenecks. Benchmarks that deliver
alignment with an artist’s day-to-day work
can help organizations make the decisions
that will guarantee the best price/perfor-
mance ratio.
Whatever way you decide to benchmark,
be aware that it is a commitment no different
from dedicating yourself to continuous edu-
cation about technologies that can contrib-
ute to the quality and speed of your work.
“Applications are continually evolving,”
says Cunniff. “A GPU that might be big
enough for yesterday’s application might
well be a bottleneck once the next version of
the application is released.” 
Bob Cramblitt is communications director for
SPEC. He writes frequently about performance
issues and digital design, engineering, and manu-
facturing technologies.
e d i t i o n 2, 2020 c gw 39
8/7/20 8:14 AM
 Transcending
Boundaries
It used to be that cutting-edge visual effects were the domain of feature films. But now,
similar work can be found on the small screen as streaming services (Hulu, Amazon, Netflix,
Disney+, and more) and premium channels (HBO, Showtime, Starz, Epix, and others) are
growing in popularity and offering extraordinary content boosted by CGI. Indeed, VFX in
entertainment appears to have no bounds. Here, we look at the complex digital work playing
out on television, computer, and mobile-device screens. This includes the VFX used to create
the enchanted world of the faeries in Amazon’s Carnival Row and the real-time design and vi-
sualization technology that has brought the amazing world of HBO’s His Dark Materials to life.
With film and television production stalled due to COVID, audiences are rediscovering these
series. To this end, CGW decided to backstep, too, and highlight this compelling work.
– Karen Moltenbrey

40 cgw e d i t i o n 2, 2020

CGW_Issue2-2020.indb 40 8/7/20 8:14 AM

 Dark Days
CREATING THE THRILLING VFX OF CARNIVAL ROW’S
FLASHBACK EPISODE
BY KAREN MOLTENBREY

C
arnival Row is an immigrant’s tale,
centered on various races who were
forced to flee their homelands when
war and invaders arrived on their doorsteps
to plunder their resources. In this story, these
venerable beings are collectively known as
the Fae, and they come from ancient, myth-
ical lands, and have settled in Carnival Row,
the seedy immigrant ghetto section of the
sprawling human city of the Burgue.
There are two primary types of Fae who
have settled here: the Pix, full-size pixies/
faeries with wings; and the Pucks, satyr-like
beings with horns and hooves. There are
glimpses of others, too: fauns, centaurs, and
the gremlin-like sprites known as Kobolds.
The Burgue is a class-based society that
resembles a dreary, grungy, steampunk-in-
spired Victorian England, and the Fae are
considered the lowest class of all here, as
the creatures – many of them working as
indentured servants – are stripped of their
dignity and their rights.
To make matters worse, a serial killer is
terrorizing Carnival Row, threatening the del-
icate balance of life in the city, where peace
Artists added digital faeries (below) to shots of actors on rigs (above).
among the Fae and humans is tenuous at
best. Keeping the peace in the area closest
to Carnival Row are constables who care
not a whit about those living here, with the Prime Video. Created by Travis Beacham and Important Looking Pirates, or ILPvfx
exception of human detective Rycroft Phi- and Rene Echevarria, the series is based on (unique environments and more in Episode
lostrate (Orlando Bloom), the only one who Beacham’s film script “A Killing on Carnival 3), plus others.
is respectful of the Fae and tries to solve the Row.” Betsy Paterson, who served as the Season 1, which dropped late last summer,
murders and keep the fragile peace. But, visual effects supervisor on Season 1, says features eight episodes. No release date has
Philostrate (Philo) has a secret of his own: the work needed to feel grounded and gritty, been announced yet for Season 2 due to
He lives as a human but was born a faerie. part of a world that isn’t ours but still feels interruptions resulting from COVID-19.
Just as damning, he has rekindled his love real and physical. Most of the series thus far has taken
affair with Vignette Stonemoss, a refuge Without question, it’s a big show with lots place in Burgue – and especially at Carnival
faerie, within this intolerant world. of VFX, which mainly involve creating char- Row. A great deal of set extension work is
Such is the premise of Carnival Row, a acters and sets, built by a number of studios, used for shots from Carnival Row, as the
web television sci-fi/fantasy drama pro- such as Image Engine (creature animation main drag consists of approximately three
duced by Amazon Studios and Legendary including the central monster), Pixomondo blocks of constructed set that extends
Television Studios, and airing on Amazon (faerie wings), Rhythm & Hues (Kobolds), vertically for only about a level and a half.

e d i t i o n 2, 2020 c gw 41

CGW_Issue2-2020.indb 41 8/7/20 8:14 AM


Film plate before digital faerie wings were added.
Wings with gray lighting.
Final comp of the CG wing added to the shot.
At either end of the street and off the side
streets is greenscreen. The bridge spanning
the river, seen at one end of the street, was
created in post, along with the train crossing
the bridge. And, uphill from this street lies
the rest of the city, which is CG as well.
However, Episode 3, “Kingdoms of the
Moon,” is a stand-alone flashback that pro-
vides history and context on Philo and Vig­
nette’s relationship, and how the refugees
have come to be in their current situation.
The story line takes viewers out of Carnival
Row and back 10 years in time, to when Philo
and Vignette first met. The episode begins
with Philo leading fellow Burgue soldiers
42 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 42
into a Fae refugee village in the kingdom of
Anoun, in the Tirnanese Highlands, where
they have come in peace and plan to
establish a defense post. The Mimasery, a
holy place, is a preserve for the religious and
educated, and now serves as a shelter for
the Fae escaping from the war.
The soldiers are wary of the magical
beings, but not Philo. He and Vignette, the
steward of the sacred library there, become
friendly and have a short-lived affair, during
which time he shares his secret with her –
that his wings were shorn off when he was a
child so that he could live a better life. When
an invasion by the evil Pact soldiers begins,
there is mass confusion and Philo and
Vignette become separated. Told Vignette
would die for him, Philo agrees to have the
message relayed to her that he was killed in
battle so she would escape to safety.
“It was a great opportunity to build a
universe unto itself within this one episode,”
says Paterson. “We’re seeing the home world
of the faeries and the war that led to the situ-
ation we see in the city during the series.”
This particular episode was submitted for
this year’s Emmys, in the Outstanding Visual
Effects category, but did not make the final
cut. It is written by Travis Beacham and
directed by Anna Foerster.
History in the Making
While the typical Carnival Row episode
contains around 200 VFX shots, “King-
doms of the Moon” has approximately 450,
encompassing partial- and all-CG charac-
ters; large, expansive CG environments and
digitally-augmented sets; as well as a big
airship battle. Image Engine and ILP were
the two lead vendors on this flashback
episode: Image Engine was tapped for the
creature work, while ILP handled the big
digital environments, the CG faeries and
their 3D wings when the faeries are practical,
and the battle.
According to Paterson, the VFX team
used practical prosthetics as much as possi-
ble throughout the season. This was true for
the faeries’ wings as well as the horns and
hooves of the Pucks as they mingle together
in the crowded streets.
In Episode 3, the village is filled with
faeries, whose wings were a combination of
practical and visual effects by ILP, a midsize
VFX facility in Sweden. The wings are
practical when the characters are walking
around and the appendages are tucked
to their bodies. Sometimes the wings had
to be refined digitally, but as soon as the
wings begin to move, they are transitioned
to CG replacements. (Nick Dudman, special
effects designer, devised a system for easily
removing the practical wings in the series,
and then a green tracking cube was placed
on the back of the costume.)
According to Niklas Jacobson, visual
effects supervisor at ILP, the CG wings had
to look very specific, with an oily, semi-trans-
parent feel. And, they had to blend seam-
lessly with their practical counterparts. Even
the animation was delicately balanced so
8/7/20 8:14 AM
 the CG versions would not stand apart, yet
the animation had to make them feel as if
they are alive and part of the character.
“It’s an incredible amount of work – the
roto alone that’s needed to remove the
practical wings and add the CG wings was
unbelievable,” Paterson notes. “We had to
roto around clothing and other characters,
and there are lots of crowd scenes in the
episode – we had to get those wings in there
between every faerie.”
For the mid- to near-ground faeries
who fly, the actresses were filmed on wires
against greenscreen. For one scene in par-
ticular, however, a giant rig was constructed
over the main courtyard set that enabled
16 to 20-some faeries to fly at a given
time, making the transition to flight more
seamless. “We weren’t separating them out
on greenscreen [for this sequence], even
though it led to a lot of rotoscoping. But, it
felt more realistic to have them walk through
the set and then lift up and fly out of there,”
Paterson adds.
When the faeries appear midground to
background, full-CG characters are used.
Pixomondo was the main “wingman” for
the majority of Season 1, and did quite a few
shots in this episode, too.
Technically more complex than the
faeries in the episode were the marroks
(werewolves), changelings that transition
from human to beast form.
“We scanned all the actors who had their
own marrok, and we worked hard to ensure
that the design of each was closely related
to the actor playing that particular charac-
ter through the use of skin tones and hair
placements,” says Paterson. Then, it was a
matter of building out the creature. Paterson
praises Image Engine’s incredible work with
the muscle systems and the skin, as these
creatures have much more visible skin than
the typically furred creature. Instead, these
have mostly human skin covering an animal
body shape.
In one particularly scene, Vignette is flying
back and forth across a ravine to help the
soldiers secure an electrical line – which
required a lot of transitioning between prac-
tical and CGI character work. Then, a CG
marrok attacks Philo on the same cliff until
Vignette rescues him.
The marroks appear in other scenes,
as well, including one wherein three Pact
soldiers strip naked in a forest and inject
CGW_Issue2-2020.indb 43
The film plate with a stand-in for the marrok. Note Vignette without her wings.
Artists added the CG marrok to the scene, as well as CG wings for Vignette.
themselves with a serum, which causes
them to transition into the beasts.
Epic Worldbuilding
The landscape in this episode is stunning,
with breathtaking hills, towering mountains,
and deep ravines. The episode was filmed
in Prague and the nearby towns, where
filmmakers captured the natural beauty of
the environment. For instance, the open-
ing sequence with Philo and the soldiers
marching to the village was shot practically
with CG extension work. The snow was real,
thanks to a well-timed blizzard, which negat-
ed the need for digital effects. For the most
part, though, the plate photography in the
episode was used for extreme close-ups.
The wide establishing shots were a
result of worldbuilding by ILP artists, who
constructed the area near the walled-off
Mimasery village including the forests and
mountains around it. “One of the more excit-
ing challenges for us was creating a believable
world and planning it out. We received some
fantastic artwork from the art department,
but we also did some concept work our-
selves,” says Jacobson. The artists generated
almost all of the environments with geometry
due to the large number of shots taking place
within the environment; there were only a
few matte paintings, used in the establishing
shots as the soldiers arrive.
Once ILP received plate photography, the
artists began the worldbuilding, carefully
constructing and then incorporating the
same type of trees and the specific type of
rocks from the live-action plates into the
digital environments. SpeedTree helped
grow the trees. “The stones and mountains
are very specific to this area, so we had to do
custom texturing work to create the specific
stones and so forth,” notes Jacobson.
Into this landscape, the ILP artists built
the village, including a massive cathedral-like
library holding sacred texts, maps, and
technology research. The bottom two levels
of the library were built practically and then
digitally extended to a soaring height, allow-
ing Vignette to fly upward while locating a
particular book. “You don’t know how high it
actually is. You never quite see the top,” says
Paterson. A spinning rig lifted up the actress
from the set piece, and the effects were
added later.
While ILP had been using SideFX’s
Houdini for many years to create effects,
this episode marks the first time the studio
migrated fully to the software as its primary
package for worldbuilding and environment
work, enabling the team to take advan-
tage of all the tools that exist within that
program.
“We have been using Maya for many years,
and we love it and still use it primarily for
modeling, animation, and so forth. But, it was
hard to assemble scenery, and it became
e d i t i o n 2, 2020 c gw 43
8/7/20 8:14 AM

A partial practical set at the Mimasery.
ILP crafted an extensive digital environment, including rock, snow, and smoke.
more difficult as we started doing more and
more big-scale environment work,” Jacobson
explains. “Houdini is a great tool for assem-
bling very complex digital builds. Also, the
integration is so smooth, having everything in
one package [for effects and worldbuilding].
Our effects TDs can do setups and provide
them to our lighters to refine certain aspects.
It’s a lean approach, getting everyone using
the same software.”
In addition to Maya, the artists at ILP
use Autodesk’s Mudbox and occasionally
Blender, as well as Foundry’s Mari for textur-
ing. They also employ asset libraries, such
as Quixel’s Megascans for high-detailed
scanned models. Compositing is done in
Foundry’s Nuke, and rendering is achieved
in Arnold. “It’s integrated into Houdini, so we
didn’t have to switch to a new render engine
while we were making this big [software]
transition,” Jacobson points out.
As stunning as the Mimasery envi-
ronments are, the most difficult set, in
Paterson’s opinion, is toward the end of
the episode, when the Pact soldiers arrive
in zeppelins and begin to shoot the faeries
from the sky as they hurl Molotov cocktails
at the soldiers. An all-out battle rages as the
sky fills with airships, which rain destruction
down on the monastery.
“We built the airships that are wreaking
havoc in the Mimasery. The fight includes
44 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 44
effects work such as fires and explosions,
as well as digital doubles and the blimps
themselves, plus the destruction,” Jacobson
points out.
Whenever we see the action from the
airship’s point of view, the shots are com-
pletely digital. “We’re seeing the top of the
mountain and the Mimasery, with all the
faeries and the Burgue soldiers in the court-
yard below, and it’s all-CG,” says Paterson.
“There’s a tiny postage stamp of practical
people in the courtyard, but everything else
– all the faeries flying out, the mountain,
the trees, the snow – it’s all created by ILP.
They built the whole mountaintop, really. It’s
beautiful.”
As Jacobson points out, ILP has tackled
some amazing creature and other type of
work since it was founded 12 years ago, but
for the past three to four years has been
called on to do larger-scale environments,
ILP orchestrated a complex battle scene using CGI.
culminating in what he calls “a milestone in
an environment build for us” on Carnival Row.
Pushing Boundaries
Some recent streaming and television
series, including Carnival Row, are pushing
the quality and quantity of visual effects like
never before – a trend mastered by Game
of Thrones, also known for its the massive
scope of work. And, Paterson is aware of the
comparisons being made between the two
fantasy series. “It’s rare that you get a show
where you’re doing this type of creature
work in addition to these environments,” she
says of Carnival Row. “And the environments
themselves are much more complex than
you usually do for television.”
Amazon Prime Video dropped the entire
first season of Carnival Row simultaneously,
though the VFX teams work on a staggered
delivery for the episodes. Still, all the teams
had what Paterson describes as a fairly
long post on the series, and even did a bit of
added editorial work at the very end. Overall,
the visual effects teams had an average of
16 weeks per episode, and since they are
being done concurrently, some of the shots
were completed quickly while others were in
process nearly the entire time. For Episode
3, however, ILP was in production for nearly
nine months. And, the results show the
effort that was taken.
The episode is a visually stunning piece,
but it also gives audiences a much-ap-
preciated backstory for these two main
characters, as well as context for the
faeries’ struggle in the series’ present day.
The Season 1 finale solves a number of the
series’ central mysteries, but it also sets the
stage for the growing inequality and tension
among the humans and the creatures as we
enter Season 2. 
Karen Moltenbrey is the chief editor of CGW.
8/7/20 8:14 AM
 Pre-production
in Real Time
PAINTING PRACTICE USES UNREAL ENGINE FOR DESIGN
AND VISUALIZATION ON HIS DARK MATERIALS
BY KAREN MOLTENBREY
S
ome might consider the pre-produc-
tion work Dan May and his Painting
Practice company do as “gameplay.”
And, in a sense, it is.
Painting Practice, a West London design
studio May founded more than a decade
ago, can be described as a previsualization/
VFX facility, offering services such as previs,
animation, VFX, production design, concept
art, and digital matte painting. It has recently
garnered attention for its design and visual-
ization work for the large sets and complex
action sequences on HBO’s His Dark Mate-
rials. And they did so using real-time tech-
nology by implementing Unreal Technology’s
Unreal Engine.
His Dark Materials is an HBO epic fantasy
drama series produced by Bad Wolf Studios
and based on the book series of the same
name by Philip Pullman, the first of which,
“Northern Lights,” was the basis for the fea-
ture film The Golden Compass (see “Animal
Control,” CGW, December 2007). The story
introduces Lyra, an orphan, who lives in a
A look at scene lighting for Trollesund.
CGW_Issue2-2020.indb 45
parallel universe where science, theology,
and magic are entwined, and where every-
one has a personal daemon, a manifestation
of his or her soul in the form of an animal.  that are practical, financial, and creative.
The feature film was ambitious, yet the
television series is even more so in terms of
its scope and scale, with a plethora of crea-
tures from witches to armored polar bears,
and mythical cities and vast landscapes
created by CG artists at Framestore (see
“Crafting Daemons,” at the end of this fea-
ture). Before that happened, though, Paint-
ing Practice provided real-time mock-ups of
various sequences that enabled the director
and producers to make decisions concern-
ing the series’ complicated sequences.
“Unreal was a great way of getting people
to physically be in a space that was not even
conceived,” says May, creative director at
Painting Practice and one of the company’s
co-founders.
As Joel Collins, executive producer and
production designer on the show, notes,
“These are very expensive sets, and on a
show like His Dark Materials, where you’re
building an entire town for one episode,
you’ve got to make really good decisions
What Painting Practice’s team created gave
us an ability to make critical judgments that
meant we could get it right on the edge of
affordable and, absolutely, on the edge of
shootable, in the sense there was no fat on
what we built.”
Painting Practice originally was set up
to facilitate production design, whether
physical or digital, but as more greenscreen
sets were being incorporated, the compa-
ny’s work on previs became more plentiful,
though it still does quite a bit of animation
as well. “Primarily we’re first-in-the-door,
working for producers, directors, or show-
runners on the vision of the show,” says May.
“We often help get them off the ground,
and then we work through the traditional
pre-production art department space, pre-
vis, and then in the last few years, with Black
Mirror and now with His Dark Materials, we’re
e d i t i o n 2, 2020 c gw 45
8/7/20 8:14 AM

Painting Practice uses real-time tech to aid
design and vis for His Dark Materials.
A look at a smoke-filled scene of Bolvangar.
very much hand-in-hand with editorial and
the effects-room postvis, which is obviously
a quite complicated and involved process
because it involves spending money on
things that might not even make it into the
cut, but has to be good enough for people to
make informed decisions about.”
In some instances, the company will do
final shots, especially if it is an environment
and within the company’s strengths, where-
by it will do final matte paintings, as they
did on the Netflix dystopian sci-fi anthology
series Black Mirror.
“We’re not looking to be a post house,”
says May. “We’re purely there to serve and
help facilitate productions or post houses to
get them through design or tricky prob-
lems, whether it’s an animation problem or
[involving] previs storytelling.”
Embracing Real Time
According to May, he had been looking at
VR and game engines for some time as way
to explore a space. He used the real-time
technology to build the night club for “San
Junipero” (Season 3 Episode 4) of Black
Mirror. “Rather than just doing a few visuals
46 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 46
and white-card modeling, I went the extra
mile and built a VR experience that the
director and producers were able to look at
and experience firsthand,” he says.
May began exploring this medium further,
examining the use of game engines beyond
gaming applications.
“His Dark Materials is full of complicated
challenges. The show has a good budget, but
with the ambition of the show, that budget
is quickly stretched. So, you have to come
up with creative solutions and find ways to
resolve situations, and this is often the tech-
nology that can do it for you,” May says.
While filmmakers like James Cameron
and Jon Favreau have been using virtual
cameras to scout locations for some time
now, the process up to now has been out of
the cost range for television. But, Unreal has
opened up that access to far more people.
Nevertheless, the integration of Unreal
Engine into the process called for a retool-
ing of Painting Practice’s pipeline. When the
company began its work with the engine, it
had difficulty getting the different software
to communicate, whereas now there are
more scripts and plug-ins available to make
the process easier and more automated.
For the most part, Maxon’s Cinema 4D
serves as Painting Practice’s backbone –
“It’s great for previs and postvis, for effects
work,” notes May. In addition, the facility
employs the Adobe Creative Suite, along
with Adobe After Effects, and Premiere for
editing and Pixologic’s ZBrush for complex
environment and character designs. The
group also uses Frame.io for collaboration
as well as delivery, archiving, and referenc-
ing, along with Quixel products (Bridge and
Mixer) and PureRef, freeware for making
giant art boards.
Worlds and Characters
In essence, Painting Practice’s Concept
team for His Dark Materials on Season 1
(Season 2’s release date is still pending)
worked with the environments and crea-
tures teams as well as with the production
designer to help determine what the worlds
and characters would look like. They would
then review that with Framestore and Rus-
sell Dodgson, VFX supervisor.
“Sometimes the directors wouldn’t come
on board until late in the process, and we’d
need to plan a lot of the sequences. So, it
was my job, really, to have a go at making
those sequences, with various outcomes
and different tone levels,” May explains.
“Then when the directors would start, we
would show them a bunch of things. Some-
times they would like the ideas or the prin-
ciple, or they wouldn’t necessarily like all the
cameras or the speed or tone. It would be
a hit or miss, but a lot of the time we got it
quite close, and then we’d make the chang-
es. But, it is really important and essential
that the directors have creative ownership
of the sequences so they can shoot some of
that previs; otherwise, [the work] becomes
null and void.”
As May points out, there were many
instances on Season 1 where the work paid
off and was used shot-for-shot, and other
times when it was used but was filmed a bit
differently although the elements and beats
were the same; alas, there were even times
when everything changed, particularly when
there were script alterations.
One of the particularly challenging sets
Painting Practice had to design for His Dark
Materials is Trollesund, the main port of
Lapland in the world of Lyra Belacqua, the
story’s heroine. For this, the team made a
8/7/20 8:14 AM
 scan of a quarry using photogrammetry
drones and used it to create a high-reso-
lution model to import into Unreal Engine.
To this, the group added additional models
and textures to complete the real-time
mock-up.
The previs enabled the director and pro-
ducers to explore the conceptual environ-
ment and make key decisions concerning
the scale of the buildings and the set. The Using Painting Practice’s Plan V.
previs also aided in the storytelling aspect,
so even though the area was called Trolle- A visual comparison of work in Unreal (left) vs. broadcast footage (right).
sund Port, not all the action had to take
place dockside, providing more flexibility
for constructing the set, which was then
extended digitally.
The environments are complicated, but
a good deal of the series’ budget from the
start went toward the complexity of the
daemons, which were photoreal, not styl-
ized. After all, the characters are a big part
of the story.
“His Dark Materials and Black Mirror
have an air of fantasy and science fiction
[respectively], but there’s also a reality, a
grounding, and that’s the space where we
fit well,” says May. “And, they have a raft
of very complicated, weird, and wonderful
challenges that we aim to solve.”
Another critical piece of previs the com-
pany provided was for a fight between two
of the series’ armored polar bears. Episode
7, called “The Fight to the Death,” involves
mortal combat between Lyra’s friend,
Iorek Byrnison, and the usurping king Iofur
Raknison. According to Jamie Childs, who
directed the episode, being able to exper-
iment with a virtual camera on a set that
did not yet exist enabled him to explore
the best camera angles to maximize the
scene’s believability.
“I wanted to walk around and look at the
bear fight because I wanted that fight to
Unreal Engine enables the director and showrunners to plan out scenes.
feel like it was a real fight being shot, not
a CG fight,” says Childs. “I could actually
A sample of a video plan involving the characters in the popular series.
do that with Unreal. I could move around
that room physically, get the camera in the
position I wanted, and see on my camera
monitor what was going on, and record
those shots and cut it together – and that
was really freeing for a director.”
According to Childs, the sensation
reminded him of his early days learning his
craft, when he would go out and just film
things to try them out, letting his creativity
find the essence of the story. “I didn’t really

e d i t i o n 2, 2020 c gw 47

CGW_Issue2-2020.indb 47 8/7/20 8:14 AM


An overview of Trollesund by Painting Practice.
The previs saves valuable shot-planning time on the complex production.
think that previs would help me do that side
of things,” he says. “I thought it might help
me with technically [determining] where
to put a camera and things like that, but it
actually made me go, ‘Right, I don’t need to
worry about any of the noise; I can actually
just go and create something.’”
In the last episode of Season 1, “Betray-
al,” a great battle plays out with exploding
airships firing at the bears and Lyra. The
Painting Practice team dropped the director
and DP into the virtual environment, some
of which “never saw the light of day due to
flooding [at the real location],” recalls May.
Initially, that scene “blew the VFX budget
out of the water because we were having so
much fun,” he adds. “Then we pared it back
to what we could then shoot, although it
was a painful process to create something
that was really interesting and fun and quite
beyond our wildest dreams, then smash it
into pieces in a reality check.”
One of May’s favorite and most success-
48 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 48
ful sequences occurred in Season 1 Episode
4, “Armour,” as Iorek retrieves his armor.
“We planned out this town and built it in
the game engine, and could visit it virtually
inside the studio, rather than having to drive
an hour and a half into the Welsh moun-
tains,” says May. “And we were able to make
complicated and very cool animations of the
bear running around and trashing the town.
We were able to get in there and discover
shots with the director [Otto Bathurst] in
the various sessions. We would make multi-
ple versions of the sequence, do an edit, and
then have different versions of the edit, with
different endings. We ended up with some-
thing that could be shot, was achievable,
on budget, and Framestore was happy with
the volume of shots. It was a collaboration
among all the partners that pulled off well.”
Plan V
Soon interest piqued concerning Painting
Practice’s proprietary tools and Unreal En-
gine setup. The company received funding
that helped push further development of
the tools, and after some fits and starts, the
visualization/effects company released Plan
V. A new software app based on Unreal En-
gine, Plan V is a bespoke studio environment
that enables artists, producers, directors,
and others involved in a production crew to
experiment with lenses, storyboards, previs,
and more.
With a simplified workflow and a user-­
friendly interface, Plan V is designed to
enable less technically inclined users to
interact in a high-quality 3D environment
so they can design worlds, sets, and scenes
for films, television, advertising, and games.
The tool also supports local and remote
collaboration.
“You can get an amazing visual look out
of [Unreal Engine] very quickly, and there’s
an incredible amount of assets that are
now available [for it], so it makes building
environments super quick. But expecting
directors and writers and others who are not
necessarily technically-minded to pick it up
is a bit of a tall task,” says May. Plan V makes
viewing the Unreal Engine scenes simple
and affordable.
Moreover, as crews continue to work
remotely for whatever reason – be it pan-
demics, travel costs, personal or professional
obligation – tools like this will become even
more valuable.
As May points out, this is such a fast-
paced industry, and it is more important
than ever to keep relevant and keep mov-
ing forward.
As of this writing, work is in progress for
Season 2 of His Dark Materials. Season 1
was a learning phase of sorts, as the group
became familiar with the virtual production
tools and cameras as well as the operation
of the engine itself and the resulting collab-
oration with the vendors. On the upcoming
Season 2, the group pushed the usage
further, creating a virtual space where the
actors could better understand the sight
lines and so forth. And as the seasons
progress, the technology will become even
more useful, May predicts, in handling the
increased fantasy elements and pushing
the storytelling.
“A lot of Season 1 was just trying to
get things to work and discovering cool
ways of working and making some great
sequences. In many ways, we made great
8/7/20 8:14 AM
 Crafting Daemons
Framestore played a key creative and collabo-
rative role in the worldbuilding process for His
Dark Materials, delivering all of the first sea-
son’s VFX and served as a key creative advisor
from the earliest stages of production.
This saw Framestore build a produc-
tion-side visual effects team at Wolf Stu-
dios in Cardiff led by senior VFX supervisor
Russell Dodgson, on-set VFX supervisor Rob
Duncan, and VFX executive producer James
Whitlam. They provided guidance, advice,
and input as to how best to bring Philip Pull-
man’s unique world to life – from feedback
on the production notes to creating the
show’s lush environments and developing
the nuanced relationships between the
actors and their daemons.  intricacies of feathers, fur, eyes, and claws.”
The result is one of the most ambitious
feats of storytelling yet seen on TV, with
over 2,000 shots of spectacular CG and
animation work bringing the richly-imagined
universe and its colorful denizens to life. “His
Dark Materials demanded an extraordinarily
high VFX baseline,” says Dodgson.   their gestures and behavior augmented,
Crafting a visual spectacle equal to
Pullman’s beloved source material drew
on a wide range of talent, and presented
challenges in terms of scope, scale, and
storytelling. “Every human character having
his or her own daemon means, theoretically,
every shot that features an actor could also
require VFX,” explains Dodgson. “This level
of work and detail is unparalleled when it
comes to television – the sheer volume of
shots requiring the inclusion of a pine mar-
ten, a hawk, a monkey, or a snow leopard
meant our lighters, animators, and CG art-
ists really had their work cut out for them.”
Framestore created more than 50
distinctive daemons as well as the bear-
like panserbjørn, effectively bolstering the
show’s overall cast with its CGI creations. To
things in previs that we probably couldn’t
make for real, ultimately. But doing those
techniques and finding discoveries were
still useful,” says May. “Even though we
were limited by our imaginations in Unreal,
we had budgets, health and safety, and a
million other things to consider. You have
CGW_Issue2-2020.indb 49
ensure emotional, realistic interactions be-
tween human actors and fully-CG charac-
ters, first-pass takes were conducted with
puppets, which helped provide visual and
interactional cues. The puppets were then
removed from set, and clean plates featur-
ing just the actors in situ were filmed. It was
then up to Framestore’s artists to conjure
forth the show’s distinctive daemons.  terms of both the quantity and quality of
“On the one hand, there was the
challenge of creating so many photoreal
creatures,” says Dodgson of the process.
“You’re building each one of these char-
acters from the ground up, which means
crafting their skeletons and musculature,
and that’s before you even get on to the
Bringing these characters to life was not
just a question of volume, however. “It was
obviously important to imbue each daemon
with its own personality,” says Dodgson. “A
deeper concern, however, was how they
then interacted with their humans, and how
reflected, or concealed what their human
was thinking or feeling.”
The scope and scale of Season 1 of the
to get realistic about what’s achievable in
terms of time and money.”
In addition to His Dark Materials, May
and Painting Practice are using the virtual
pre-production process on other shows,
which are under NDA at the moment.
“I think we’re still in the early stages about
show meant Framestore drew on skills from
across the entire company, with the work
split between offices in London and Mon-
treal. “It was a team effort on a grand scale,”
says Fiona Walkinshaw, Framestore’s global
 
managing director, film. “We’re no stranger
to huge projects and tight deadlines, but His
Dark Materials really feels like a landmark in
VFX shots TV audiences witnessed. It’s a
perfect example of how film and high-end
TV are converging: There’s a heightened de-
mand from clients and audiences alike for
VFX work that would look as good in your
living room as it would at the IMAX.” 
As well as allowing the company’s artists
to flex their creative and technical muscles,
the work also tapped another of Frame-
store’s core skills: the ability to get inside
a piece of existing IP and breathe life into
complex creatures, characters, and settings
that exist in the hearts and minds of a great
many loyal fans. 
“This isn’t a world we’re building from
scratch – it’s one that millions of readers
feel deeply invested in,” Framestore CEO
William Sargent points out.
how it’s going to be used,” May adds.
Meanwhile, the series recently received
two BAFTA awards, including one for visual
effects. 
Karen Moltenbrey is the chief editor of CGW.
e d i t i o n 2, 2020 c gw 49
8/7/20 8:14 AM
 Spinning a Yarn
NETFLIX’S THE WILLOUGHBYS – NOT YOUR TYPICAL TALE

BY KAREN MOLTENBREY

Images ©2020 Netflix.

50 cgw e d i t i o n 2, 2020

CGW_Issue2-2020.indb 50 8/7/20 8:14 AM

 T
hroughout our lifetime, we have
read fairy tales of all kinds, nearly all
of which have happy endings. The
Willoughbys, an animated film based on the
book by the same name, is not one of them.
“If you love stories about families that
stick together, and love each other through
thick and thin, and all end happily ever after,
this isn’t the film for you,” says the narrator,
a world-weary cat, in the opening of the
animated film. “This family’s story is weird,
hidden away from the modern world in their
old-fashioned home.”
Indeed, the cat’s perception of the story
is accurate. However, when looking at the
film as a whole, “unique” seems to be a
more fitting description than “weird.”
Unique Story
Inside the big, old-fashioned home filled
with antiques is a history that goes back
a long way, a family legacy of tradition, inven-
tion, creativity, and courage, as depicted in
the portraits hanging on the walls. The family
magnificence has been passed down from
generation to generation (like their promi-
nent red, bushy facial hair) – that is, until the
current one.
The house now sits in the middle of a
modern, bustling city, closed off from the
rest of the world. Living amid all the books,
paintings, and heirlooms is the latest family
of Willoughbys: Mother (Jane Krakowski) and
Father (Martin Short), who are selfish and
lovestruck. They share the space begrudg-
ingly, their children: Tim (Will Forte), Jane
(Alessia Cara), and the Barnaby twins (Sean
Cullen). While love is an amazing thing, the
The Willoughbys, directed by Kris Pearn, is based on book by Lois Lowry.
Images ©2020 Netflix.
CGW_Issue2-2020.indb 51
parents only have love for each other; they do
not extend it to the children, who they see as
nuisances. Better not seen and not heard.
The children are mistreated. They are
neither fed nor clothed. Never is a kind word
offered to them. One day the children find
an orphan outside the gate to their home.
They want to keep it, but their parents forbid
it. The kids find a seemingly perfect place for
the baby: at a candy factory. Convinced they
would be better off as orphans themselves,
the siblings come up with a plan: entice
their self-absorbed parents to take an exotic
vacation to dangerous locales around the
world. As the pièce de résistance, they add
the caveat “no children allowed” to their fake
brochure, ensuring their parents’ interest. It
works, and off the parents go.
Happy to be “orphans,” the children’s
staycation is interrupted by a nanny (Maya
Rudolph), who the parents have hired to
watch the kids. Eventually, the children
warm up to her, that is, except for Tim, who
remains mistrusting.
Feeling guilty about tricking their parents
(even after Mom and Dad sell the house so
they can afford to continue their journey),
the children enlist Nanny and a generous
candymaker, Commander Melanoff (Terry
Crews), to help them build a candy dirigible
so they can rescue their parents before
tragedy befalls them. However, the children
sneak off in the airship by themselves and
fly off to the “Unclimbable Alps,” the last
stop in their parents’ trip. The children
succeed in their mission, following a trail
of Willoughby-red yarn up the mountain
and arriving in the nick of time before the
lovebirds freeze to death. But the ungrate-
ful pair continue their selfish ways and
commandeer the flying contraption the
kids built, abandoning the children on the
mountain top to a frozen fate.
But maybe the story has a bit of a happy
ending, after all, which has nothing to do
with the parents. Instead, Nanny and the
Commander rescue the children, who are
truly orphans now, and they, along with baby
Ruth, the infant at the gate, become a family
– a perfectly imperfect family.
This anti-tale, based on the book by
Lois Lowry, was brought to life in Burnaby,
Canada, at a greatly expanded Bron Anima-
tion, a division of Bron Studios, a Canadian
motion-picture company. The film is written,
produced, and directed by Kris Pearn (Open
Season, Surf’s Up, Cloudy with a Chance of
Meatballs). In fact, Pearn’s varied back-
ground in the animation industry (animator,
story artist, character designer, writer, and
director) made him an ideal candidate for
this project, which took on a homemade
aesthetic, due to his experience in 2D,
stop-motion, and 3D animation.
Producer Luke Carroll optioned the book
in 2014 after listening to an audio version on
a family vacation. Pearn signed on, and work
began in earnest two years later and finished
just about two weeks before North Amer-
ican went into lockdown due to COVID-19.
“Taking the tropes of animated films and
having the opportunity to play with them to
tell a story that I think is different in a way
that celebrates the independence of kid
logic and hopefulness was an interesting
challenge,” says Pearn.
e d i t i o n 2, 2020 c gw 51
8/7/20 8:14 AM
 The Willoughby children long to be rid of their parents.
Unique Design
The artwork from the book is sparse, giving
production designer Kyle McQueen a clean
palette to work from. The design he and the
filmmakers chose is certainly atypical, one
that feels homemade. “The stop-motion
influence in The Willoughbys came from this
idea of an old-fashioned story – kids growing
up on books rather than the Internet – and
so we really wanted to dive into that from
a visual standpoint and create something
that didn’t feel digital,” says McQueen. “We
wanted a world that really felt tactile and
visceral, like turning the pages of a book and
smelling the ink.”
Adds Pearn, “I wanted the audience to
think they could go to Michael’s [craft store]
and buy all the materials to make this movie.”
The characters are disproportionate in
shape. But what stands out the most are
the textures. The Willoughby family, as
generations before them, have big red hair
that looks as if it is made of yarn, a notion
that is emphasized by the fact that Mother
is obsessed with knitting. Nanny also sports
a big head of hair, only hers is dark brown,
curly, and heart-shaped.
Likewise, all the clothing looks handspun,
too, like you would see on a stop-motion
character – making them look like minia-
tures. That is, except for Tim, who is wearing
a too-small explorer-like outfit he outgrew
many years ago.
“When I pitched the idea to Kyle Mc-
Queen, he immediately got excited by the
notion of boiling down the shapes to simple
cartoon principles with heightened textures,
almost in the vein of stop motion,” says
Pearn, who spent part of his career at Aard-
man Animation, known for its stop motion
productions such as Shaun the Sheep. “Ev-
52 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 52
erything in stop motion is handmade, so we
started wondering, what if everything in this
world was handmade? Hence, countless
frames [are] overflowing with textures and
toy-like components.”
Unique Animation Approach
As big a role as stop motion played in
defining the film’s look, it is Pearn’s 2D
animation background, with its hand-drawn
approaches and pose-to-pose animation,
that held sway when it came to making the
characters move.
“Early on we looked at 2D pose-to-pose
animation, and I challenged the animators
to create a movement style for each char-
acter based on who they were,” says Pearn.
For example, Tim is very still, almost like a
grown-up. Jane, meanwhile, represents the
future, she just wants to escape. She’s like a
bird, fragile but tough, which played into the
way she moved through the use of arcs.”
While the film is informed by a stop-mo-
tion aesthetic and 2D animation principles,
the world of The Willoughbys is neither in
actuality. Rather, it is 100 percent CGI. And,
“tricking” the computer into believing oth-
erwise during the creation process was the
biggest challenge Pearn and the crew faced.
The Willoughby history of greatness skipped Dad’s generation.
Textures were key to achieving the tactile
aesthetic of the characters. The way Pearn
used the cameras also provided a stop-mo-
tion feel. To this end, McQueen devised an
environment that could be captured without
a lot of digital moving  cameras, and Pearn
decided to use minimal motion blur, citing
Hal Ashby’s ​Harold &  Maude​and three-cam-
era sitcoms as inspirations (see “Inspiration,”
page 55). “When we do move a camera, you
feel it,”  says Pearn. “We tried to pick cameras
and do setups that gave us a lot of width. We
use a lot of depth of field so that your eye
always knows where to go. It does kind of feel
like miniature  photography. And that’s a big
part of the stop-motion appeal.”  
Creatively, the lack of camera motion also
gives the sense that the children are stuck in
the house. “I wanted to shoot the house like
it was a practical set, like a sitcom. Locking
the camera allowed us to give the audience
that three-camera setup feeling where
there are long takes, you trust the acting
and let the composition do the work,” Pearn
explains. “On the flip side, when we unpin
the camera, it drives a story point. When
the children leave the house, we let the
camera dolly up, because that telegraphs to
the audience that we’re now loose and out
8/7/20 8:14 AM
 Meet The Characters
Jane (Alessia Cara)  Nanny (Maya Rudolph)  on his doorstep. Once the siblings start
Jane Willoughby, the family artist, is a joyful
and curious 12-year-old. Though outnum-
bered by her three brothers, she is full of
ideas, opinions, and daydreams, and is eager
for her voice to be heard.
A huge-hearted optimist, Nanny is hired to tinkering in his high-tech factory, Colonel
take care of the Willoughby children while Melanoff has softened into a generous spirit
their parents travel. Once she realizes that and doting dad.   
they’ve had a sheltered upbringing, Nanny
introduces the kids to new food, music,
and views, becoming their tour guide to
modern life. Always bursting into melodies
and giving out hugs, Nanny emerges as the
comedic backbone of the story.  

The Cat (Ricky Gervais) 

Since the adventure unfolds from this four-
legged narrator’s viewpoint, the filmmakers
Tim (Will Forte)   designed the elements on-screen to resem-
As the eldest ble a collection of colorful, handmade toys.
Willoughby child, At first, The Cat is a jaded onlooker prowling
Tim fancies him- through the Willoughby home, pausing to
self their leader. Mother (Jane Krakowski) lick himself and grumble about the latest
He has firm & Father (Martin Short)  human happenings. Yet he roots for the
beliefs about how The selfish Willoughby parents pay little kids and
his family should attention to anyone except each other. increasingly
act to maintain Hidden away in a mansion full of antiques, feels like part
its legacy, which  Mother and Father entertain themselves of their
causes some ar- with old-fashioned crafts like knitting and family, so
guing with Jane. A building  ships in bottles —
​ until their kids much so that
brave, energetic, trick them into booking a vacation. On that he creates
and scholarly  trip, they a diversion
14-year-old, Tim has endless potential ​— ​if learn to reunite
he can give up being in charge, listen to his about the them with
siblings, and have a little fun. Internet Nanny. 
and
Barnaby A & B (Seán Cullen)  decide to Ruth​(Brian Drummond) 
Whether finishing each other’s sentences, sell their Pink-cheeked baby Ruth will scale any sur-
solving complex math problems, or building home face and gobble every snack. After Mother
gadgets from scratch, the young twins from afar. and Father refuse to take her in, the giggly
always act as one Willoughby. Clever go-get- youngster brings the Willoughby siblings
ters, Barnaby A & B love learning and follow- and Nanny into Commander Melanoff’s life.
ing instructions word for word. They admire She also sends the kids on their first trek
their older siblings very much, often acting into the city, foreshadowing their eventual
as their referees.    globe-trotting
journey. 
Commander Melanoff
(Terry Crews) 
A world-famous candymaker whose face
beams from bus and TV ads, Commander
Melanoff is actually quite lonely. New ac-
quaintances tend to be intimidated by his
size, uniform, and baritone bellow, but not
Nanny or Ruth ​—​the infant he discovers

e d i t i o n 2, 2020 c gw 53

CGW_Issue2-2020.indb 53 8/7/20 8:14 AM

 Cat’s Tale
Director Kris Pearn wanted The Wil-
loughbys to be a fairy tale. To this end,
he decided to have the cat (voiced by
Ricky Gervais) narrate, enabling the
director to push the animation a little
The film contains a homespun aesthetic throughout.
further over the top.
“The movie starts off from the
point of view of this external charac-
ter, the cat, who’s looking through a
window. It’s the cat’s tale, told from
its point of view, so it might not be
100 percent real,” Pearn explains.
“That gave us a sense this could be
a once-upon-a-time sort of world.
That let us pivot into the world of
metaphor. And because we had
these heightened textures, we had
permission to then push the choices
we would make in terms of posing
and pushing our characters toward
that almost broad Cartoon 101/
Chuck Jones approach with simple
silhouettes.”
Nevertheless, the animators
always maintained gravity, the sense
that the world was real. If a character
fell, he or she would hit the ground
and likely react accordingly.
“We wanted to have that phys-
icality,” says Pearn. “I wanted it to
seem like there’s always bones in
the characters, but there are bones
that have a lot of loose joints when
they hit big poses. And yet, they are
not rubbery.”
54 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 54
into the world. When the children make the
choice to come back and stay at the house,
the camera is locked again.”
To further implement the less-is-more
approach of simple, graphic character de-
signs, the  crew coined a term related to T
​ he
Willoughbys​’ cinematography: variable frame
exposure. Today, standard features are filmed
at 24 frames per second, with every frame
displaying a different image. With variable
frame exposure, the filmmakers showed
certain images for one,  two, or three frames.
“You might be giving them fewer images per
second, but it never came at the cost of less
information,” maintains Aniket Natekar, one
of the film’s lead animators.
Pearn explains this further. “Instead of
splining the animation so that it’s full-frame
range, 24 fps, we would do it on twos for
the characters, and then we would do our
effects on threes and fours. The feeling that
I was trying to achieve was similar to when
[filmmakers] would use film and shoot their
primary layer and roll it back, and do the ef-
fects on a pass-through. I wanted things to
feel like they had a little bit of an off-timing.
Of course, the computer really likes it when
things make sense. So, trying to figure out
how to work with the software and the pipe-
line to give us the robustness of the textures
that we were after, and the timing we want-
ed, was really a challenge. But, we had some
really smart people working on the film. Russ
Smith [VFX supervisor] was able to get in
there and add scripts and break the widgets,
and get [the software] to do what we were
creatively chasing. Plus, it was a small studio,
which enabled us to be really collaborative.”
Bron Animation was already established
as an Autodesk Maya-based studio, which
the Willoughbys team used as well. The
artists also used Pixologic’s ZBrush for mod-
eling. In addition, they used Maya’s XGen
geometry instancer for populating polyg-
onal mesh surfaces with primitives, either
randomly or uniformly placed, to generate
the family’s signature red hair as well as the
cotton candy elements.
“By controlling the movement as opposed
to straight simulation, we could make the
hair feel designed, and we could respond to
what we were learning from the textures,”
explains Pearn. “Establishing this style of
movement also gave us cost savings in
various conditions, such as rain, snow, and
wind. The graphic quality of the yarn and the
rich texture added to the handmade feeling
by not overreacting to the FX elements,
which we wanted to feel comp’d over the
movement, rather than embedded.”
Unique Effects
While the imagery in the movie’s foreground
is 3D, the backgrounds were matte paint-
ings, again supporting that homemade feel.
“I loved the idea that behind a certain layer
of depth, there’s a painting,” says Pearn.
“Also, we had to commit to the locked
camera idea so it felt like a sitcom. I found
that soundstage feeling very attractive to
the story.”
A good portion of the film takes place
inside the house, where Pearn found the
children’s library to be an especially difficult
environment due to the extreme detail and
the difficulty in lighting it so the viewers’
eyes knew where to look. “I wanted the
room to feel rich. It’s sort of a metaphor;
it’s full of all the ideas bubbling in the twins’
heads,” he says. Grading was also challenges
to ensure the characters stood out among
all the detail.
The snow near the end of the film was
also challenging in terms of establishing the
right combination of 2D and 3D. It took a lot
of trial and error, Pearn says, to get the snow
to feel like it was applied to the scene, but
also gave the audience the sense that the
world is very cold.
“We had this amazing artist, Helén
Ahlberg, who was our effects lead. She was
a 2D animator by training, but had that
8/7/20 8:14 AM

The animated film was crafted at Bron Animation.
superpower where she could step between
the 2D and 3D worlds,” Pearn adds.
According to Ahlberg, effects are usually
accurate calculations and simulations, but
for this feature, the filmmakers wanted the
effects to feel homemade, to fit with the
overall aesthetic. “That was definitely a big
and extreme challenge for our team,” she
says. “We would use the 2D mind-set of
building them by hand, but we would do it in
the computer.”
The production designer wanted fire
to look like cutout paper and snow to feel
like 2D confetti flying around. To this end,
Ahlberg and the visual effects artists drew
inspiration from a number of places for the
work, from stop motion, to paper cutouts,
to children’s illustrations, focusing on the
craftsmanship and then mimicking that
in the computer using Maya and SideFX’s
Houdini. For rendering the FX, the group
used Mantra within Houdini. (The lighting/
comp team used RenderMan within Found-
ry’s Katana for rendering and Foundry’s
Nuke for compositing.)
Their approach varied somewhat
depending on what was needed, but it all
started by first sketching out the effects
by hand for timing. “We had to really think
creatively and go outside the box a bit,” says
Ahlberg. “We didn’t come up with any new
groundbreaking technology. It was a matter
of taking a unique approach that would
give us very strong silhouettes and graphic
shapes on all the effects.”
For instance, when crafting fire, Ahlberg
wanted something iconic that felt like fire
but didn’t have big, fluid movement like the
element usually does. “We played around
with triangular shapes and would hand-
sculpt them to look like they were made
out of paper. And then we put a light that
flickers inside to symbolize the chaos that
comes from fire,” she adds. “But when you
look at it, you know it’s fire. We’d even have
some things that were on twos or threes or
CGW_Issue2-2020.indb 55
fours, instead of the normal ones for effects,
so they wouldn’t stand out too much.”
In addition to fire, the group tackled other
elements. For smoke or dust or clouds, they
tried to make them look like cotton or even
spun cotton candy. And for water, Ahlberg
didn’t feel like fluids would really fit the style
of this film. “So, we thought more about the
material and played  around with reflec-
tive Mylar strips, just deforming them and  Chuck Lorre kind of time,” explains
breaking them apart,” she says. “Splashes
would break into little pieces of paper
instead of droplets.”
There is also a rainbow made out of
cotton candy. The effects department also
handled some of the cloth, such as a carpet
or flag. And, the group was in charge of
paper electricity, bubbling soda acid fields,
and lava.
Unique Ending
In most coming-of-age stories like this,
there would be a fairy-tale ending. The
parents would see the error of their ways,
and the family would reunite and live hap-
pily ever after. Well, that was not the case
here, obviously.
Still, the kids do get a happy ending, but
not the one the audience expects. And the
parents, well, that’s another story with… and
abrupt ending.
Likewise, viewers probably did not expect
the unique aesthetic of this film. “We really
have tried to create a hybrid world that looks
like a  cross between CG and stop motion.
This is a movie that looks like nothing else
out there,” Natekar says.
Adds McQueen: “So much love and care
went into this film, and I really hope that au-
diences find it and love it as much as we do.
It’s such a weird, funny, emotional film, and I
think it’s very unique in the animation space.”
We couldn’t agree more. 
Karen Moltenbrey is the chief editor of CGW.
Inspiration
The Willoughbys is a CG-animated
film, though it looks as if it is a hybrid
production of 3D, 2D, and stop
motion. Also, it uses techniques from
dual mediums, infusing the camera
style and more used on various tele-
vision sitcoms within this feature-film
production.
“Visually, one of the things early
on I really wanted to achieve was
the collision of filmmaking styles. I
wanted the time that the kids spend
in the house to feel like a sitcom,
where we really lock the camera and
have three-camera setups and long
takes. And, I wanted the rat-tat-tat
of the dialog to remind people of that
Kris Pearn, director. “And then when
they leave the house, the camera
unpins and the colors change; the
idea is that outside of the gate is a
movie. That kind of collision between
those two worlds was something we
really got excited about early on in
the process.”
Pearn was particularly influenced
by TV shows he grew up watching,
such as Three’s Company, Cheers, and
All in the Family, whereby the char-
acters might be wrong about their
opinions of the world, but there’s an
underpinning of optimism.
As for the mixed aesthetic, Pearn
was also influenced by his stop-­
motion background, but also by his
2D animation experience. “I still love
the choices that come from pose-to-
pose animation, like when you watch
Chuck Jones or the old Disney hand-
drawn films like 101 Dalmatians or The
Jungle Book, that feeling that there’s
a hand behind the pencil,” he says.
Despite the stop-motion and 2D
influences, Pearn never considered
creating The Willoughbys as anything
but a CGI film. In this vein, practicality
reigned, as Bron Animation already
had a 3D pipeline in place and was
able to draw on the abundance of 3D
talent in the Vancouver area to create
this film.
e d i t i o n 2, 2020 c gw 55
8/7/20 8:14 AM
 Classroom Disrupted
TEACHING THE NEXT GENERATION OF ANIMATORS DURING COVID
BY KAREN MOLTENBREY
I
n this era of COVID-19, educational
institutions have had to rethink the way
they teach. In the spring, schools had to
move to online/remote learning with just a
moment’s notice. With many animation and
visual effects schools/departments current-
ly on summer break, institutions have been
using the time to look beyond their earlier
Band-Aid approach and more thoroughly re-
search and evaluate their teaching methods
and directions as the fall semester quickly
approaches.
We spoke to a number of animation/
visual effects schools to find out what types
of alterations they had to make to finish up
the spring semester, and how that affected
the students and curriculum. We received
various responses. We also asked what kind
of changes they are looking to adopt for the
fall and what kind of impact this ongoing
crisis might have on student learning.
Indeed, plans are under way to provide
the best experience under the circumstanc-
es. (Keep in mind, the schools were queried
in early July, just about the time when severe
outbreaks were re-occurring. So, schools
may be forced to alter their plans once again
before fall semester begins, as the pandem-
ic situation remains fluid.)
 
SCAD
Savannah College of Art and Design (SCAD)
is on the quarter system, not semesters,
with its traditional academic year span-
ning Fall, Winter, and Spring quarters, with
Summer quarter optional. Earlier this year,
two weeks before the Spring quarter started,
the university announced the need to move
all courses and instruction online due to the
virus. SCAD wanted to be sure students had
that time, as well as the first week of class, to
decide whether they wanted to take classes
in a completely online environment, or drop
courses and receive a full tuition refund.
Furthermore, if a student was dissatisfied
with an online course experience this past
spring, SCAD has offered the option to re-
take that course for no additional tuition and
56 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 56
in-person when on-ground courses resume.
“Our successful spring and our ambitious,
thoughtful plans for fall are not really about
following some mythical ‘playbook.’ This is
simply SCAD at its finest: inventive, strategic,
positive, collaborative, transformative,”
says SCAD president and founder Paula
Wallace. “Very soon, our beloved campuses
will reopen and students will swarm joyfully
through our doors and classrooms once
again. It is a new day in academia, and SCAD
is poised to shine on and keep shining,
thanks to the resilience, light, and love of this
heroic learning community.”  students receive individual attention in a
In March, SCAD launched a new ini-
tiative, Guests and Gusto, a Zoom series
programmed exclusively for SCAD stu-
dents during the university’s virtual Spring
quarter 2020. The dynamic series, which will
continue, features new weekly content that
represents all of SCAD’s 40-plus top-ranked
degree programs through illuminating con-
versations, surprise drop-ins, and interactive
games.  Guests have included:  author and
New York Magazine art critic Jerry Saltz;
award-winning producer and songwriter
Tricky Stewart; TEDtalk’s head of television
Juliet Blake; film director and producer
Morgan Neville; actor Alan Cumming; and
GQ creative director and author Jim Moore.
Students also engaged with industry leaders
in their individual classes, which led to many
students receiving job offers and internships.  wave. Twenty years ago, we took art and
Meanwhile, SCAD commencement 2020
was a high-energy, live-streamed event.
SCAD celebrated its newest alumni with a
personal digital commencement experi-
ence, featuring speaker Alicia Keys and host
Rachel Brosnahan.
 For the Fall 2020 quarter, SCAD will offer
real-time on-ground instruction, real-time
virtual instruction, and anytime online
instruction. These methods all individual-
ize the acquisition of knowledge and the
advancement of student outcomes.
SCAD will resume real-time on-ground
instruction at SCAD Atlanta and SCAD
Savannah this fall, inviting students back
into the university’s award-winning studios,
classrooms, and historic buildings while
adhering to health and safety protocols.
A SCAD task force, in close consultation
with outside health and safety experts, has
created plans for maintaining appropriate
social distancing, with enhanced cleaning of
high-touch areas, as well as recommenda-
tions on PPE requirements where appropri-
ate, especially in situations where full social
distancing is challenging.
SCAD classes are already small in keeping
with the university’s mission to ensure all
positively oriented environment. The average
class size is 21 students, and lecture-style
classes are capped at 35 students. 
Real-time virtual instruction, what most
SCAD students experienced during the
spring quarter, will continue alongside on-
ground learning this fall. 
SCAD eLearning allows students to work
independently, explore content, participate
in online discussions, and move at their own
pace on a Monday-through-Sunday sched-
ule. SCAD eLearning offers 24 graduate and
undergraduate degrees in everything from
animation to interior design.
 “Because SCAD has always been com-
mitted to the ‘what’s next’ philosophy and
execution in professional practice, I see our
students excelling in this digital and virtual
design education online; over a decade ago,
SCAD was one of the first universities to use
REVIT as a required part of the curriculum,
now it’s an industry standard,” says Anthony
J. Cissell, AIA, chair of Architecture and
Urban Design.
Virtual reality and the online collabora-
tive studio is going to be a standard of the
architectural practice of the future, and
this is where our students, who are digital
natives, are going to be professionally ready.
“Here at SCAD, we are already working in the
virtual reality realm through our programs
in Motion Media, Digital Effects, and Game
Design. Now we’re exploring how to bring
8/7/20 8:14 AM
 those cutting-edge resources and strategic
partners to our Architecture students to po-
sition them as leaders in this virtual model of
professional practice,” says Cissell.
Academy of Art University
Academy of Art is but one of the many
institutions that swiftly responded to
address the pandemic issue this past spring
by locking down its San Francisco campus
and moving classes online and virtually. It
may seem like such a drastic shift in learning
environments, but the truth is that it was
a transition that required no fanfare on the
school’s part because it was an environment
already set up and functioning in place.
The university’s online education program,
established in 2002, allows its students and
faculty to carry on as normal in their classes,
albeit with certain modifications. Make no
mistake, our online courses are designed to
be just as rigorous and comprehensive as
those taken on campus, says Catherine Tate,
director of the School of Animation and
Visual Effects.
“It will require just as much effort and
commitment to make the grade. To put it
simply, our students haven’t missed a beat....
It’s a challenge that puts a lot of systems
to the test, but it’s a challenge that we’ve
long been prepared to ensure our students’
education is not interrupted,” she adds. For
instance, the school’s Spring Show was held
virtually this year and was a success.  The Fall 2020 semester at SVA is sched-
For now, Academy of Art University will
continue to offer its remote and virtual
classes for the Fall 2020 semester. Howev-
er, the school will be opening its labs and
workshops for studio course homework
along with its residence halls and dorms.
“We will be taking precautions to protect our
campus community by utilizing virtual class-
es for lecture, demo, and critique (which we
feel is an unnecessary group risk to perform
in-person), and allow students to reserve
facility time for on-site labs, studios, and
workshops,” says Tate.
In addition, Academy of Art has been
forced to accelerate the StudioX process,
a student-run, faculty-mentored experi-
ence that emulates a real-world animation
and VFX production environment where
students work with real clients, on real films
and games, with real deadlines – and real
problems to solve. The situation also is
enabling the school to offer more flexibility
CGW_Issue2-2020.indb 57
and options to online and on-site students,
including face-to-face contact via Zoom with
instructors and other students, Tate notes.
School of Visual Arts
School of Visual Arts (SVA) suspended
in-person classes beginning Wednesday,
March 11, and began remote learning on
Monday, March 16. “In the midst of this
global health crisis, it was critical for us to
continue on with the semester so that stu-
dents’ progress towards their degrees went
uninterrupted,” says Jimmy Calhoun, chair
of Computer Art, Computer Animation, and
Visual Effects.
SVA, like every single free-standing,
accredited college of art and design, made
the best of it via Zoom and the school’s on-
line learning platform, Canvas. Once remote
learning began, the school used RemotePC
to provide students with remote access
to the facility. In addition, many software
companies graciously provided the stu-
dents at SVA with licenses for the remain-
der of the semester. “We’re very grateful for
how generously and quickly they provided
support,” says Calhoun.
Calhoun also applauds the dedication
of SVA’s faculty to unwaveringly deliv-
er their course content, even for studio
disciplines, and who will continue to meet
and engage with students virtually until
restrictions are lifted.
uled to run September 28 through Decem-
ber 22, spanning 12 weeks, with extended
class durations. The spring 2021 semester is
scheduled to start on the previously estab-
lished date of January 11, and is to run for the
full 15 weeks.  when it is safe to do so in the future.
Upon reopening, SVA will follow govern-
ment guidelines and best practices in higher
education for social distancing, which could
include some or all of the following measures:
limiting class size, wearing masks while inside
campus buildings, and limiting the size of
campus functions and activities in accor-
dance with state and local requirements.  school representative, it is also studying the
Rising Sun Pictures
There was minimal disruption to the courses
and the curriculum when COVID-19 hit at
Rising Sun Pictures Education in Australia.
It took some time to pivot, but the time lost
to set up the new classrooms was added to
the end of the course, plus a few extra weeks;
submission deadlines were extended also.
“Students were still able to receive the
experience they expected, which was
incredible given the circumstances,” says
Anna Hodge, training & education manag-
er, Rising Sun Pictures. “With protocols put
in place, students said they felt safe at-
tending classes during the pandemic. They
also enjoyed the amendments that were
made to the course due to social distanc-
ing, and were grateful for the experience
and opportunity.”
RSP wrapped up the semester with
students presenting their work in the RSP
theater to key artists. Pre-COVID, the FX &
Lighting and Compositing & Tracking stu-
dents viewed one another’s presentations
together. However, with restrictions placed
on room numbers, the groups were split into
two separate sessions. Zoom was set up
so the students in the other session could
still watch the presentations, albeit from
another room.
Two weeks after the students complet-
ed scheduled classes, RSP invited other
local visual effects companies to come
and view the students’ work and provide
individual feedback. According to Hodge,
the students valued the networking experi-
ence and appreciated the feedback about
their projects. Already, she has received
follow-up emails from employers making
queries about the graduates.
LA Film School
The Los Angeles Film School was able to
maintain its graduation ceremonies earlier
this year, albeit in a virtual setting with the
plans to have a larger in-person graduation
 
Meanwhile, campus classes were re-
vamped to be 100 percent online as of this
interview, in addition to the school’s existing
fully online programs, and will remain this
way until it is safe for students to return
to campus in line with the CDC and local
governmental guidelines. According to a
guidelines of the DGA, major studios, and
the local IA and SAG AFTRA, and will make
adjustments to the curriculum to support
the new reality of making films in a post-
COVID-19 world. 
Karen Moltenbrey is the chief editor of CGW.
e d i t i o n 2, 2020 c gw 57
8/7/20 8:14 AM
 Ahead of the Curve
ANIMATION MENTOR HAS LONG PRACTICED ONLINE LEARNING FOR ANIMATION STUDENTS
BY KAREN MOLTENBREY
W
hile many animation and visual
effects colleges and education
institutions have had to quickly
pivot to virtual/online learning due to the
novel coronavirus, not so for Animation Men-
tor. The highly regarded animation school, in
fact, was founded on the premise of distance
learning 15 years ago, pairing up its all-remote
student body with mentors, experienced
working professionals in the industry.
Since its founding in 2005 by three
working animators – Bobby Beck, Shawn
Kelly, and Carlos Baena, from Pixar Anima-
tion Studios and Industrial Light & Magic
– Animation Mentor has developed and
perfected a curriculum and infrastructure
specifically tailored to remote students.   been using Zoom for all its online teacher/
The structure of the school’s courses and
the nature of its community have always
been virtual, so the online transition that
many other educational institutions were
forced into earlier this year did not alter
Animation Mentor’s own teaching process-
es. The school believes in smaller class sizes
(which it limits to nine students per instruc-
tor) and experienced, tech-savvy mentors.
It also runs on its own proprietary platform,
which keeps students and mentors engaged
and encourages its community to make
friends and connections.
At a time when many schools are
scrambling to shift its education model from
in-person learning to online learning due to
COVID, it was pretty much business as usual
at Animation Mentor, at least from a curricu-
lum and platform standpoint. “We made
sure our students knew we were available if
they were in need of help, and we also made
sure to put additional resources in place to
help students manage their focus time,”
says Beck, CEO and co-founder of Anima-
tion Mentor.
Even though Animation Mentor wasn’t
negatively affected by stay-at-home orders
like other schools, some of its students’
lives were heavily impacted by the virus. To
this end, the school staff made themselves
58 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 58
available to listen and help find solutions for
struggles the students may have encoun-
tered. This included providing assistance for
issues related to financial struggles, or even
lack of motivation in the face of uncertainty.
They also instituted extra resources on a
weekly basis, including an office hours-style
live session with animator and mentor Paul
Allen. This gave students an additional way
to connect, feel supported, and talk through
any issues (creatively or otherwise) they
may be facing.
Nevertheless, there were some changes/
precautions taken by the school to keep
the education process running smooth-
ly. For instance, Animation Mentor has
student meetings for years, but recently
implemented additional security features to
ensure that the learning experience wasn’t
interrupted. “Zoom became the go-to for
online meetings this spring, and with that
additional scrutiny, instigators bombarded
the Zoom platform. The security features
we enabled ensured that only our students
and mentors could join each meeting (for
Q&A),” Beck says.
Out of the gate, Animation Mentor
embraced online instruction. Ambitious,
yes, but the founders believed it to be the
right move, even amid many warning against
an Internet business at a time when the
dot-com bubble had just burst. “Despite
the bubble, we knew that we wanted to be
online. We were part of awesome creative
communities like CGChar, and we wanted
to take things to the next level. We wanted
to bring animation to people in their homes
and bring the top talent in the animation
industry directly to people so they didn’t
have to move across the globe to access
them,” says Beck.  theless, Animation Mentor recognized and
The founders also liked the fact that there
weren’t any models for online animation
schools, allowing them to define what that
could look and feel like.
“We wanted to make online feel more
personal than a brick-and-mortar school,
and we wanted people to make friends and
connections they’d have for life,” says Beck.
“The cool thing is that those friendships
could be across continents and come from
different cultures, bringing their voices and
their stories to the world of animation.”
The founders are as confident in their
decision now as they were 15 years ago, as
the advantages to this model are many. For
instance, it allows people from anywhere in
the world to learn alongside equally passion-
ate peers, and it allows people who are more
introverted to have their voices be at the
same level as those with louder voices. In
addition, students can see other students’
work and mentor feedback on the virtual
campus, regardless of the class they’re
enrolled in.
“An open campus means they can learn,
not just from their own class group, but
from other students as well. This facilitates
a sense of mentorship, with the advanced
classes helping newer students gain their
confidence and footing,” says Beck.
Not only do students join Animation
Mentor to learn a new creative skill like
animation, modeling, storyboarding, visual
development, and so on, but they also
come out with so much more, according to
the school. They build confidence in them-
selves, and they make incredible, lifelong
friendships and connections, both with
their mentors and the Animation Mentor
staff. There have even been over two dozen
marriages from students who have met
through Animation Mentor. 
Of course, there are challenges to online
learning, as many other colleges are just
now discovering, such as accountability
and potential feelings of isolation. Never-
addressed these early on, and from the
beginning, focused on making the school
more than just a place to learn, but a place
to come together and connect as well. As
a result, students feel engaged, even when
8/7/20 8:14 AM
 they’re spending the majority of their time
alone in front of their computers.
The school also stresses the importance
of deadlines and following the mentor’s
(director’s) instructions. More than just
learning the art of animation and other
artistic subjects, the school also teaches
students good work ethics and habits. “We
want students to be prepared for the studio
environment, and having a strong work ethic
is a big part of that,” says Beck. 
The core teaching model at Animation
Mentor has remained the same over the
years – generally a small class size to each Work by Antonio Vieira, from the 2019 Animation Mentor showcase.
mentor, so that students get more indi-
vidualized attention. The curriculum and
technology, however, are always evolving.
The school is constantly looking at how its
classes are performing, and make routine
adjustments to the curriculum to ensure the
best learning experience. 
Furthermore, the school is constantly
evolving its technology and platform, creating
custom-developed tools to help support its
teaching model. This includes feedback tools
where mentors (and any other community
members) can directly draw over and com-
ment on student work. Animation Mentor
also built a Web-based pipeline for students Work from Cinthia Mussi, also from the same showcase.
so that they can begin learning studio work-
flows while in class, as well as gaining a host
of other useful features from that system.
The core teaching model at Animation
Mentor typically involves some pre-recorded
material that covers the key learning goals
for the week. That’s followed by a live Web
meeting between the mentor and students.
Students get an assignment to complete by
the end of the weekend, and then the mentor
will record a video critique of the assignment,
noting what’s working and what could be im-
proved. This continues each week as student
learning and growth continue.
Work from Daniel Vargas, from that showcase as well.
Animation Mentor offers four terms
per year – Winter, Spring, Summer, and
Fall – and all of the courses and workshops
start at the beginning of each term. Since try go from a very low percentage of remote over the last six months. “People are giving
the stay-at-home orders began in March, workers to 100 percent remote workers in online learning a real chance, and they want
Animation Mentor has seen an increase in record time. It’s something we’ve always to see if it really is ‘as good’ as their brick-
enrollment, something it attributes to the believed in, as it gives artists the opportunity and-mortar school. We hope that people
fact that people are at home with additional to have more balance and gives the industry are pleasantly surprised. It’s not something
time to focus on animation. the opportunity to tap into talent that they we had to piece together overnight,” Beck
Will this trend toward remote education may have had to pass on previously due to points out.  
and even remote working continue across visa restrictions,” he says. 
the industry? Beck believes so. “Firstly, it’s As far as Animation Mentor is concerned,
been interesting to see the animation indus- it has seen an increase in the student body Karen Moltenbrey is the chief editor of CGW.

e d i t i o n 2, 2020 c gw 59

CGW_Issue2-2020.indb 59 8/7/20 8:14 AM

 Hybrid Drama
THE BLACKLIST’S OUT-OF-THE-BOX PRODUCTION STRATEGY DURING THE PANDEMIC
BY KAREN MOLTENBREY
I
f necessity is the mother of invention,
then adversity can be considered its
father. Both are fitting descriptions of the
tack producers of the NBC crime thriller The
Blacklist took after filming abruptly ceased
when stay-at-home orders were instituted
because of COVID-19.
Unlike some television series that had
already completed production and even
post-production for the season, The Blacklist
still had a number of episodes to go before
its Season 7 finale. In fact, The Blacklist was in
the midst of shooting Episode 19 of 22 when
the coronavirus lockdown went into effect.
What to do? The network already had
renewed the series for an eighth season, and
audiences expect an edgy show, especially
one as dramatic as The Blacklist, to close
out the season in a thrilling, suspenseful
way. Realizing that completing the planned
season-ending Episode 22 was out of the
question, the producers believed Episode 19
would make the second-best ending. The
60 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 60
problem was, that episode was only halfway
through filming in New York.
Nevertheless, by thinking outside the box
and using CG animation to fill in the missing
segments, the showrunners were able to
accomplish their goal with a very unique and
suspenseful episode that leaves audiences
counting the days until the series returns.
This is not the first time that CGI has
saved the day for films and TV series that
have encountered unforeseen obstacles
during the middle of shooting, mostly involv-
ing actors who have unexpectedly passed
away before filming was completed (Paul
Walker in Furious 7, Oliver Reed in Gladiator,
Brandon Lee in The Crow, and Nancy March-
and in The Sopranos, to name a few). But,
what happens when the situation adversely
affects not just one person, but the entire
cast – and crew? The solution was CGI. Only
this time, entire sequences were animated
in a comic-book/graphic novel style –
including actors as well as environments –
and integrated into the live-action that had
already been shot.
Creative Problem-Solving
The Blacklist stars James Spader as Ray-
mond “Red” Reddington, a former US Navy
officer-turned-high-profile criminal who
agrees to work for the FBI to capture (or kill)
the world’s most dangerous operators and
terrorists on the government’s “Blacklist”
(in exchange for immunity) – all the while
following his own secret agenda. The twist:
This antihero will only work with the young
agent Elizabeth Keen (Megan Boone), who
may or may not be his daughter.
Season 7 Episode 19, called “The Ka-
zanjian Brothers,” finds the FBI Task Force
investigating an accountant with criminal
clientele, and to find him, they try to locate
a pair of brothers who are hired to protect
him. Meanwhile, the cat-and-mouse game
between Liz and Red is coming to a head,
as is Liz’s relationship with her mother, the
8/7/20 8:14 AM
 dangerous Katarina Rostova, as Liz is forced series, the showrunners had an aesthetic in recent years. But, it was not quite ready for
to choose sides in the deadly game between mind; they also pointed to animated films this prime-time application. To this end,
Red and Katarina. such as those by Richard Linklater (Waking the Proof team in Atlanta began develop-
Shooting in New York was only partially Life, A Scanner Darkly). “They were fond of ing special shaders for the imagery within
complete when the cast and crew were sent rotoscope-style animation, and were familiar Autodesk’s Maya.
home. During brainstorming sessions, series with the work at Proof,” says Matt Perrin, “Because we were pitching for final
creator Jon Bokenkamp and executive pro- senior visualization supervisor at Proof in Lon- delivery on a show for broadcast rather than
ducer John Eisendrath mulled over a number don. “That was kind of the jumping-off point for previews, we had to redevelop our toon
of options, including a live cast reading to fill for us, and we developed the look from there, shader to add more detail to the models and
in the missing shots. Then they pondered creating our own toon look that was more to get a closer likeness to the actors with
whether a hybrid live-action/animation sophisticated [than what we use for previs] hatching on the faces,” says Coglan.
approach would work. After all, The Blacklist and more in line with what they wanted.”
has been made into graphic novels in the The work was indeed a departure from Animation Meets Live Action
past – not such a large leap given the show’s the typical animations Proof usually does for Due to time restrictions – the team of 30-35
pulp-fiction feel to the various blacklisters visualization work. Yet, most of the anima- artists and TDs (20-plus in London and
and even Red himself, who dons a signature tors there, including Perrin and Coglan, had 10-15 in Atlanta) had just over five weeks
trench coat and fedora. Eisendrath knows at one time or another worked on projects to complete the work – there was no time
Ron Frankel, owner of visualization company for final delivery. And as both point out, to devise an entirely new process. “We had
Proof, Inc., and approached him to see if the although the visualization work is intend- to rely on what we had done all along, and
idea was doable and to seek general advice ed as a pre-production planning tool, the use that as a model and adapt as we went
and information due to their unfamiliari- quality has grown quite sophisticated in along,” adds Coglan.
ty with the medium. After a great deal of
discussion, Bokenkamp and Eisendrath
decided to proceed, and hired Proof to The toon-style animation meshed well with the series’ spy drama theme.
help them end the season with Episode 19,
which would be fleshed out with animation
by Proof, whose own staffs in Atlanta and
London were under stay-at-home orders
and working from home.
According to Proof producer Patrice Avery,
all the characters with unfinished parts
became animated, which included all of
the main and featured casts. “It was pretty
much, ‘Here is what we’ve shot so far, and
this is what we need.’ So, sometimes we were
transitioning from a close-up of Liz to an
animated Liz,” she explains. “It was a matter
of making it work with what we had [filmed].
It was an interesting ride, that’s for sure.”
The episode begins with an introduction
by the cast explaining the reason for the
hybrid direction. Then throughout, there is
an animated morph from the live-action Proof generated CG versions of the actors with unfinished parts in the episode.
into the CGI. And within the animation itself,
comic-book chyrons are sometimes used
to provide context and to emphasize subtle
emotional beats.
The use of the two methods is split nearly
50/50, with approximately 22 minutes of live
action and 21 minutes of animation.
“We all knew the aesthetic Jon Boken-
kamp was going for and found it fairly
straightforward to adapt their ideas,” adds
Adam Coglan, visualization supervisor at
Proof in London.
Because of the comic-book/graphic novel

e d i t i o n 2, 2020 c gw 61

CGW_Issue2-2020.indb 61 8/7/20 8:14 AM


In addition to characters, Proof also matched live-action environments.
One immediate adaptation involved
a remote workflow due to the at-home
restrictions. As a result, the animators had
to get used to communicating remotely
through Zoom and WhatsApp. They also
relied heavily on Teradici technology to log
in to the Proof studio network after the
OpenVPN they had been using was unable
to handle the added workload. “The process
went extremely well. The IT guys rebuilt our
infrastructure to enable us to work from
home, something we never really did before,
so we could work on some other shows we
had going prior to The Blacklist,” says Coglan.
The team also had to ensure that the final
output was void of all the shot information
that appears on visualizations. “Logging
and tracking that information without the
HUD that we usually depend on was a new
process for us,” says Perrin. Additionally, they
used Apple’s ProRes video compression
format for the final export, which is not very
common in the feature-film world.
One of the biggest changes came in the
workflow, as the project schedule for The
Blacklist had to be greatly compressed,
which led to more overlap among the
departments – asset building, R&D, layout,
animation. “There was no time to wait for
one department to finish before we put it
through to the next,” explains Coglan. “So,
for a long time, the producers had to look
at grayscale characters that didn’t look
anything like their respective actors.”
Because the imagery for The Blacklist was
so heavily stylized, that meant redeveloping
the asset pipeline, and all of the Maya-based
shaders and lighting rigs the group typically
used had to be revised to achieve the toon
look and the hatch lines. The artists still used
Maya for the animation and most of the ren-
dering, along with Maya, Pixologic’s ZBrush,
and Adobe’s Substance Painter (formerly
62 cgw e d i t i o n 2, 2020
CGW_Issue2-2020.indb 62
from Allegorithmic) for texturing and sculpt-
ing. They also used Adobe’s After Effects for
the warped transitions from live-action to
animation. An Adobe Photoshop filter was
used to toon-ify the backgrounds.
Characters with Character
To create the characters, an asset builder
at the London location pulled as many of
the bodies, clothing, and other items from
its generic character libraries as possible,
and then built the remainder within Maya.
While that was ongoing, proxy versions of
the characters (grayscale stand-in mod-
els that didn’t resemble the actors) were
employed for the animation, to get a feel for
the scenes, since “we were literally building
everything from the ground up,” says Perrin.
There were no storyboards – the animators
had to do all the staging themselves.
Later, using detailed cast information
(photos, body measurements, and so forth),
the modelers continued the character builds
so they resembled the respective actors.
The finished models were then sent to the
Atlanta-based studio to be toon-ified and
hatched; the hatching was often augmented
with hand-drawn lines on the main char-
acters’ faces, as it’s a look that requires a
degree of finesse, notes Perrin.
Afterward, the models were passed
back to the London team, where the client
signed off on them and they were placed
into the scenes.
The scenes progressed at different rates,
with some in the animatics stage while
others reached final animation. “And [the
producers] understood that. Often they
were still looking at their hero characters
in grayscale, but they trusted that every-
thing would look right at the end. They
knew what to look for – shot design, story
progression – with the details to be filled in
near the deadline,” Perrin points out.
Initially, the animators weren’t going to
incorporate much lip sync, but Coglan and
Perrin determined they could use Proof’s
previs rigs to do some basic work in that
regard. “The rigs were not set up to do full-
on facial lip sync, but the animators were
doing the lip sync as they were going along
anyway, so all we had to do was switch it
on in the end, which pleased the Blacklist
group,” says Coglan.
Meanwhile, the audio files were recorded
by the actors themselves from their homes.
The artists also had to toon-ify the back-
grounds. Although, the hybrid production
enabled the producers to be more flexible
with the planned environments. “When John
and JB (Jon Bokenkamp) committed to do-
ing the comic-book [style], they committed
completely; it inspired them to get a little
more exotic with some of their locations
and play with things they perhaps hadn’t
thought about when they had the restraints
of shooting [the DC-based show] in New
York,” says Perrin. “All of a sudden, a scene
that took place in a dimly lit apartment was
now at the Washington Mall, which was a
big build for us.” There were more crowd
scenes, too, and even one sequence wherein
a helicopter lands in a forest clearing as the
accountant attempts his getaway.
While some scenes enabled the artists
to flex their creative muscle, other scenes
that had already been filmed required them
to translate those physical locales into CGI,
which they did with the help of blueprints,
drawings, and even references in past
episodes. For instance, the artists built the
medical facility where Liz’s grandfather lies
in a coma using set drawings from the crew.
“They had already filmed some shots in that
location, so it was important for us to match
the continuity as we transitioned to the
comic-book world,” says Perrin.
It’s not easy to transition production on
dime, but that is exactly what the actors,
producers, crew, and Proof animators did
when faced with the sudden shutdown.
And, they were able to find a successful
solution thanks to the use of computer
graphics, even embracing the flexibility the
medium provided in terms of action and
storytelling. 
Karen Moltenbrey is the chief editor of CGW.
8/7/20 8:14 AM
 Quest for Reality
A PRIMER ON LIGHT FIELD TECHNOLOGY
W
e increasingly have been hearing
about light field technology as it
pertains to simulating the “ulti-
mate” visual experience. But, what exactly
is light field technology, and why should we
care about it?
In the purest sense, a light field is “just”
a mathematical function to describe light
flowing in all directions through all points in a
volume. When applied to cameras, where a
conventional image is a captured represen-
tation of light intensity, a light field image
captures both the intensity and the direction
of that light. When applied to display devic-
es, the combined function of intensities and
directions means the captured subject can
be visualized from any perspective.
Here, Dan Ring, head of research at
Foundry, provides a primer on this technol-
ogy in a Q&A with Computer Graphics World
Chief Editor Karen Moltenbrey.
Why is light field technology important?
A conventional lens-and-sensor system
collapses all of that dense light field data
down into a two-dimensional image. This
loses information and effectively makes final
choices at capture time (about things like
focus, parallax, and even camera posi-
tion) that cannot later be manipulated in
post-production, and also cannot be re-cre-
ated for display to viewers later.
But, if the richer data could be captured
and displayed to the user, it would provide a
viewer with a natural sense of parallax from
head movements, and also allow a viewer’s
eye to converge and focus naturally into
different depths within the scene.
Deferring these photography decisions
on capture allows greater freedom
when visualizing.
The viewing experience would then be
much closer to ‘being there,’ in a way that
stereo 3D displays have only ever approxi-
mated. In addition to the viewing position
and ability to focus on different depths, the
added visual richness afforded to the viewer
is in view-dependent surfaces. For example,
CGW_Issue2-2020.indb 63
metallic or speculative surfaces will appear
far more ‘real.’
Which segments of the industry will
benefit most from light field tech, and in
what way?
Firstly, light field tech could benefit live-ac-
tion on-set capture, where the additional
data could be widely exploited in post-pro-
duction, particularly in visual effects where
it could allow for easier or higher-quality
integration of elements.
Secondly, if in the future display devices
were to become available at scale, post-pro-
duction could pass the data through to
the final display to create an unequaled
viewing experience. Well before this became
practical or affordable for home use, it could
be a compelling part of premium cinema, in
a more persistent way than stereo 3D has
managed. Light field displays could also
have an impact if they were used instead
of LED panels in virtual production envi-
ronments, as they would enable natural
parallax, lighting, and focus.
What about any segments outside of
our industry?
Light fields already have uses in industrial
applications, where the extra data can be
used for detailed inspection (http://raytrix.
de/inspection/, for example). Visualization
of products in the industrial and automotive
design space will particularly benefit from
light fields, where the value of more physi-
cally correct light transport is important.
Light field technology is not new, is it?
The mathematical and physical concept of
a light field dates at least back to the work
of Michael Faraday in 1846 and has been
continuously expanded on since then. The
practical applications in photography are
also not new and have been used experi-
mentally for at least 100 years.
It seems that suddenly we are hearing
more about light field tech – why is that?​
Dan Ring
Practical digital light field capture is a com-
plex engineering problem as well as a com-
plex computational problem. The explosion
of available compute, storage, and network-
ing over the last 20 years has brought the
field to the cusp of being viable.
Visualizing light fields had been a problem
up until around 2012, when techniques for
compressive sensing of light fields on the
GPU were developed and used by MIT,
MPI & Nvidia, for example. This enabled
more content to be delivered to multi-layer
display devices at higher spatial and depth
resolutions, and started putting light fields
under people’s eyes and driving interest in
the technology.
How important are light fields to achiev-
ing a realistic visualization experience?
Simply put, if you could fully capture the
complete light field of a space and fully
display it back later to a viewer, it would be
visually indistinguishable from the reality.
In practice, of course, there will be limits on
resolution and dynamic range, which make
it an imperfect experience, but in principle,
it still has the potential to deliver a visual
experience that can’t ever be achieved by
two-dimensional images.
A key part of a ‘realistic visualization
experience’ is obviously the actual experi-
ence. Light field display technology offers
e d i t i o n 2, 2020 c gw 63
8/7/20 8:14 AM
 a different way of reproducing a captured
or approximated light field than, say, a
VR headset.
With the current volumetric display de-
vices, the visualization is delivered through
a physical display in the same world as the
viewer – they’re not being asked to be taken
out of their current world.

What are some of the specific obstacles

to achieving light fields, and why are
they important?
The volume of data is orders of magnitude
greater than conventional imagery, and the
weight and size of capture equipment is sig-
nificant compared to current camera tech-
nology. While there aren’t any compelling
reasons to support the technology (benefits
on set, benefits in post-production, or
benefits to a viewer), it is highly unlikely that
adoption will happen any time soon. The
movement of such large amounts of data
is impractical, and the energy requirements
(and heat output) of prototype displays are
very large.
Assuming the above problems of storage,
computation, and transport can be solved,
there also needs to exist the production
pipelines for working with light fields, from
capture to post-production and delivery. On
the post-production side, deep images –
which are an image-based, camera-view-de-
pendent projected volume approximation
– have been used for VFX and animation for
a while now. Even though the data rates are
significantly lower than light fields, they are
still substantial enough to make working with
deep images challenging.
Again, the investment required and the
timescales to produce are huge.
How does one go about solving
these issues?
Like all technology advances, we see a lot
of dogged persistence by innovators in
capture technology and in display technol-
ogies. Once one of these finds a niche that
A light field array built by the Saarland Informatics Campus.
is commercially viable, there’s a chance that
niche can expand. We may not need to solve
capture to post-production to viewer all in
one go.
When we finally do overcome these is-
sues, what will the result look like?
I’d start from the answer to the above
question, ‘How important is light fields to
achieving a realistic visualization experi-
ence?’ If we have that, then there are some
first things we can think about that might be
practical enterprise applications before the
tech could one day evolve to home use:
 Large displays used for on-set back-
drops, replacing LED walls with environ-
ments that have the correct perspective
from multiple viewpoints and allow natural
focus pulls into synthetic environments.
 Large displays used for future cinema,
to maintain a premium viewing experience
for big-screen entertainment.
 Large displays used for live events.
Why do light fields have the potential of
eliminating viewing gear in VR?
Digital CAVE environments aren’t new but
could be greatly improved with light field
display panels, removing the need for head
gear while greatly improving the experi-
ence. This isn’t likely to be viable in a home
environment, but we think that the switch
from headsets to an immersive wall display
could be very compelling, adding a sense of
immersion without necessarily needing a full
wrap-around experience.
In either case, the ambition would be to
move away from stereo display technology,
which is a very poor approximation for a
human visual system.
Is there any other information about
light fields you would like to highlight?
Foundry has been developing light fields re-
search as part of SAUCE, a collaborative EU
Research and Innovation project between
Universitat Pompeu Fabra, Foundry, DNeg,
Brno University of Technology, Filmakad-
emie Baden-Württemberg Animationsin-
stitut, Saarland University, Trinity College
Dublin, and Disney Research to create a
step-change in allowing creative industry
companies to reuse existing digital assets
for future productions.
The goal of SAUCE is to produce, pilot,
and demonstrate a set of professional tools
and techniques that reduces the produc-
tion costs of enhanced digital content – in
particular, targeting the creative industries
by increasing the potential for repurposing
and reusing content as well as providing
significantly improved technologies for digi-
tal content production and management.
For more insights into light fields and
related topics visit Foundry’s Insights Hub at
https://bit.ly/insightslightfields. 

2020, Issue 2: COMPUTER GRAPHICS WORLD (USPS 665-250) (ISSN-0271-4159) is published bi-monthly with special additional issues in January and July resulting in 8 issues
per year by COP Communications, Inc. Corporate offices: 620 West Elk Avenue, Glendale, CA 91204, Tel: 818-291-1100; FAX: 818-291-1190; Web Address: info@copprints.com. Pe-
riodicals Postage Paid at Glendale, CA, 91205 & additional mailing offices. COMPUTER GRAPHICS WORLD is distributed worldwide. Annual subscription prices are $72, USA; $98,
Canada & Mexico; $150 International airfreight. To order subscriptions, call 847-559-7310.
© 2020 CGW by COP Communications, Inc. All rights reserved. No material may be reprinted without permission. Authorization to photocopy items for internal or personal use, or
the internal or personal use of specific clients, is granted by Computer Graphics World, ISSN-0271-4159, provided that the appropriate fee is paid directly to Copyright Clearance
Center Inc., 222 Rosewood Drive, Danvers, MA 01923 USA 508-750-8400. Prior to photocopying items for educational classroom use, please contact Copyright Clearance Center
Inc., 222 Rosewood Drive, Danvers, MA 01923 USA 508-750-8400. For further information check Copyright Clearance Center Inc. online at: www.copyright.com. The COMPUTER
GRAPHICS WORLD fee code for users of the Transactional Reporting Services is 0271-4159/96 $1.00 + .35.
POSTMASTER: Send change of address form to Computer Graphics World, P.O. Box 3296, Northbrook, IL 60065-3296.

64 cgw e d i t i o n 2, 2020

CGW_Issue2-2020.indb 64 8/7/20 8:14 AM

CGW_Issue2-2020.indb 65 8/7/20 8:14 AM
CGW_Issue2-2020.indb 66 8/7/20 8:15 AM