(Reference) VFX Compositing From The Phygital Laboratory - User Manual I

266 TH E PH YG ITA L L A B O RA TO RY 2022
VFX
COMPOSITING
Xinyue(Grace) Ji, Jinqi(Jerry) Jiang, Luying(Niki) Xu
Visual effects is the process by which imagery is created or

manipulated outside the context of a live-action shot in filmmaking
and video production.
Compositing is the process of combining multiple images to form
a single, cohesive image.
VFX compositing seamlessly integrates digital assets with live-
action footage to bring together the final shot of a film or game.
This technique applies in both large scale industries, like movies,
TV series and commercials, and independent scale
In this tutorial, we will be introducing the key factors, and general
workflow of making a compelling VFX composited footage, given
the most accessible softwares and hardwares.
WINTER I D E A S E N T E R TA I N M E N T S T U D I O 267
CASE STUDY
(INDUSTRY)
With a total of 1,200 visual effects shots in “Blade Runner 2049” — that comes out to
1:45 of the movie’s total running time of 2:43 — Oscar-winning VFX supervisor John
Nelson and his team logged in major hours to go a step beyond the 1982 original
movie’s legendary sci-fi look.
After deciding on the environment, the production team designed the brutalistic style
architecture which takes on a bit of the scene of the first movie. They found some
places in Mexico City fitted the scene critierias, so that they went for on-site shooting
on a cloudy day.
During compositing, they put in the footage and dropped the street, creating the
sunken canyons. So not only did they make the top of the city higher, theye made the
bottom of the city lower, and created the future city of Los Angeles.
GRACE JI, JERRY JIANG, NIKI XU
Aerial View VFX Compositing

in BLADE RUNNER 2049
Academy Award Winner

BEST ACHIEVEMENT IN VISUAL EFFECTS
BAFTA Award Winner

BEST SPECIAL VISUAL EFFECTS
Satellite Award Winner

BEST VISUAL EFFECTS
VES Award Winner

OUTSTANDING VISUAL EFFECTS
“Gravity is a hybrid: it’s partly live-action, but is similar to [an

animated feature] in many respects,” says animation supervisor
Max Solomon. “Very large parts of it are fully CG.”
At the start of the pitch process, it wasn’t clear just how much
of Gravity would have to be computer-generated. Although early
tests were done using more traditional methods such as shooting
actors on wire rigs and building physical sets, it soon became clear
that the challenges of simulating weightlessness and characters
spinning off into the darkness could not be overcome practically.
As in so many previous cases, digital effects became the silent
star of the film.
VFX Breakdown
GRAVITY
CASE STUDY
(INDIE)
Ian Hubert debuted the first episode of Dynamo Dream, his sci-fi web series. The
whole project is built in Blender, a free and open-source 3D computer graphics
software toolset from a core group in Holland. He mastered VFX compositing,
using Blender from years of self teaching. His success is not only based on him
mastering the technical skills, but more importantly to understand the concept
of world building when it comes to the production of a compelling and cohesive
project.
Green screen scene being composited into the

final production
In this case study, the content creator, StudioBinder, walked

the audience through the steps of recreating a scene of a large
production movie, Inception. The significance is to help people
understand how to create visually compelling scenes at indie
scale and at low budget, which can be easily adopted in studio
production. First of all, he laid out all the props and assets required
Inception VFX Breakdown – Dream World Cafe Scene |
for the shoot, and a proposal. The second step is crucial, which
is to decide the physical and digital assets for the convenience
of future compositing. Finally, After Effects is the major software
used to complete the task. In this case study, the
Recreating The Scene
ANALYSIS
At its core, VFX compositing is the bridge between live-action
video footage and digital assets. It adds the minor details to make
the digital effects look like they’re a part of reality.
Imagine you wanted to create an explosion in an action movie. The

film crew would shoot a background, the animator would create
a digital missile, and the effects artist would add the explosion
effects for the detonation.
The VFX compositor would take all these pieces and put them
together to create a seamless shot. They would layer the pieces,
make color corrections, adjust depth, and add any minor details
needed to make all the pieces into one big, audience-wowing shot.
GLOSSARY
PERSPECTIVE
The base of setting up the work space for compositing.
LIGHTING
Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do
eiusmod tempor incididunt ut labore et dolore magna aliqua.
HDRI
An important to when it comes to creating a integrated lighting
environment, and realistic reflective materials.
COLOR
Color correction, and mood setting for production.
GRAIN
Adds to cinematic quality to the footage.
RESOLUTION
Consistent resolution is vital to the integration of different types
of media contents.
PERSPECTIVE
Perspective Matching is the process of determining the position of a camera when it took a photograph/
footage so that 3D graphics can be combined with that photograph/footage and ‘match’.
CASE STUDY IMAGE(S)

Sources (include one of these annotation boxes per
Image Reference Title (TOP, BOTTOM ETC.)
LIGHTING
Direct/indirect affects the visual perception the most
General or ambient lighting is intended to light up a room in its entirety.
image)
HDRI
HDR expands the range of both contrast and color significantly. Bright parts of the image can get much
brighter, so the image seems to have more “depth.” Colors get expanded to show more bright blues,
greens, reds and everything in between.
COLOR
Colorists color graids the whole movie giving it an artistic look which was decided by a director. LUT
stands for “look-up table. A LUT is a tool that lets filmmakers, editors, and colorists save particular color
grades as a template.
GRAIN
Although shot by high resolution digital cameras, grains are added to mimic the metallic silver particles of
traditional films.
Sources (include one of these annotation boxes per
Image Reference Title (TOP, BOTTOM ETC.)
RESOLUTION
Consistent resolution is the key to cohesive compositing.
image)
HARDWARE
The range of hardware is vast for VFX Compositing. From
smartphones to black magic camera, it depends on the different
quality of footage that you want to work with.
LIST OF REQUIRED HARDWARE
iPhone
DSLR Camera Canon 5D Mark II
Blackmagic Camera
Tripod
Stabilizer
Microphone
SOFTWARE
As we are introducing the VFX Compositing workflow at indie
scale, we focus on two major softwares that are on the top of the
list below, and some other softwares as experimentation.
LIST OF REQUIRED SOFTWARE
Cinema 4D S24
Adobe After Effects 2022
Blender
Nuke
Natron
Fusion Studio
Adobe Premiere 2022
Creating VFX in C4D iPhone video shooting Blackmagic Camera with
WINTER
With hand-held stabilizer

I D E A S E N T E R TA I N M E N T S T U D I O
PHYSICAL SETUP DRAWING/DIAGRAM

277
COMPOSITING PIECES
Caption
Caption
Step by Step -
Cinema 4D
In this section we will be looking at how to bring the footage in
Cinema 4D and then composite the footage with your 3D models.
This will include some of the modeling and texturing work.
1. INSTRUCTION ON TITLE COLORS
RED TITLES: Steps that are important and recommended to

follow.
BLUE TITLES: Steps that are not essential and can be skipped.
2. LIST OF REQUIRED SOFTWARE
Cinema 4D R23 or newer (R25 in this case)

Octane Render for C4D Available as a monthly subscription
3. LIST OF TEMPLATE FILES
Image Sequence A folder including image files

VFX_Template.c4d C4D file
Through step 01 to step 09, you will

be able to set up the basic 3D modeling
space, as a preparation for the modeling
of your compositing process.
Camera Tracking
01 Consult with the camera tracking team for the details about camera tracking.
This breakdown starts with a finished camera tracking file in C4D.
Open the c4d file and we will see the interface as below.
Relocate the File Path

02 Click on ‘Motion Tracker’, and then click on the ‘...’ button, open the image sequence
file, and select the first image of the entire sequence and click open to relocate the files
of the image sequence.
1.Click here
2. click here
Select the first image of the entire sequence and click open
(Skippable) Import Still Image

03 Follow this stepif you want to match the perspective of your camera with a still image.
Create an octane camera (you can press ‘shift’ + ‘C’ and then type in the command to
search for it), right click and then choose ‘Camera Calibrator’ in the tracker submenu.
Go to https://youtu.be/C0976dRXG_A, from 2:35 to 14:40 to learn how to use camera

calibrator.
Click here
Click here
*The camera calibrator is a different workflow with camera tracking.

Projection Mapping - Create A Material

04 To setup a model space where you can place your 3D assets in the ‘real
footage’, you should use projection mapping.
Click on the ‘Motion Tracker’, then click on ‘Create Background Object’, this will both
create a ‘Background’ and a material which is the footage itself.
1. Click here
A material is created
2. Click here
Press ‘Shift’ + ‘F2’ to open material manager to see the materials
Projection Mapping - Projection

05 Apply the material you just created to an object as the backdrop/blockouts.
Click one the material tag, in the ‘Projection’ drop down menu, choose Camera Mapping.
Then set your camera object (Solved Camera) in where it says ‘Camera’, and then set
the film aspect to the same aspect ratio (16:9) as your footage.
Go to https://youtu.be/C0976dRXG_A, from 16:00 to 19:40 learn more details.
Click the material tag
Create an object, e.g. a cube, and

apply the material to the object
(by dragging the material)
1. Choose ‘Camera Mapping’
2. Click this small button and

then click on ‘Solved Camera’.
*A good way to avoid your mapping material getting
shades from other light sources is to set your
projection material as an emissive material. 3. Change the film aspect here
depending on your footage’s aspect
ratio, in this case it’s 1.778 (16:9).
Look at the right camera

06 To be able to see how the mateial is projected onto the objects, click on the
camera at the top center in your viewport, then choose ‘Solved Camera’.
The projection mapping should

look the similar way
Blockouts & Modeling

07 You can simply apply the camera mapping (projection mapping) to whatever
object you want, and match their form and size with objects from the footage
as blockouts. There will be cloud points in a tracked c4d file. Follow them to model
the blockouts.
Go to https://youtu.be/C0976dRXG_A, from 16:00 to 19:40 to see how to set up the

blockouts.
Model the basic geometries and

put them on the right place.
*Increase the number of subdivision if the material looks distorted.

Shadow Catcher Material

08 In order to get the right shadow without any textures, you may want to use a
shadow catcher material.
Go to https://youtu.be/OeUgmej5rqE to learn how to use the shadow catcher.
(Skippable) Bake the Projection

09 In some cases you may want to bake the projection mapping to your object, so that
later you can move around that object with a fixed material.
Select the material tag of the object and right click ‘Generate UVW Coordinates’,
this will create a new UVW tag and the type of the projection of that material will be
changed to UVW mapping
The new UVW tag
Click here This one is changed to

UVW so the material
no longer follows the
camera mapping.
Through step 10 to step 15 you can

start putting your own assets/models in
this3dmodel!
Gather Assets
10 You would like to put your own models into this project file after setting up
the blockouts, either by modeling by yourself, or downloading it from websites like
‘SketchFab’.
In this case, you may also use the ‘Asset Browser’ in C4D (shortcut ‘Shift’ + ‘F8’), and
then search for the models like ‘scaffolding’.
Click here in C4D R25, or press ‘Shift’ + ‘F8’,

or press ‘Shift’ + ‘C’ and search for ‘Asset’.
Search here and
grab the object
you want
Match the Lightings

11 In order to get shadows, you will need to set the lightings. In this case, create
an ‘Octane Daylight’ (‘Shift’ + ‘C’ to search for it), then press ‘R’ to rotate the direction
of the light until the shadows of the blockouts matches the shadows in the origianal
footage.
Rotate the direction

Unmatched shadows Matched shadows
of the daylight
Balance the Contrast

12 You would like to balance the contrast of the lightings and shadows. You can
then click on the ‘Octane Daylight’, and set the parameters as you wish, or as the
following screenshot.
Click here
(Skippable) HDRI
13 You may want to use Hdri as your backdrop and overall lighting.
Create an ‘Octane Hdri Environment’ (‘Shift’ + ‘C’ to search for it), and choose the Hdri
file you have prepared, either downloaded or produced.
Go to https://youtu.be/etOkgjrTfa8 to learn more about how to create and HDRI map.
Click here to choose

your HDR file
Alpha Channel
14 You may want to see the ‘background’ object in STEP 03 appear in your render
viewport, or to get rid of the default white/grey background, or in some other cases
to get a png file that has a transparent background.
Open ‘Octane Settings’ (‘Shift’ + ‘C’ to search for it). In the ‘Kernels’ tag, scroll down and
tick the check box of ‘Alpha Channel’.
The higher the sample rate is, the better quality your image
is going to be, and the longer time the rendering takes.
For a good quality, it is recommend to be over 1000.
Export Settings
15 You would like to render the composited footage after everything has been
put together in the project file.
Open the render settings. Under the ‘Output’ tag, set the aspect ratio as 1920 * 1080 (or
1920*816 if the film ratio is 2.35:1).
Under the ‘Save’ tag, set the file path of the renderings. The format is recommended
to be png as an image sequence. You can later put them together in Premiere or AE.
Click here to open

render settings
Set the output file path
Set the resolustion to 1920*1080
Tick the checkbox

if you need alpha
channel
*Thiswhole chapterisskippable
‘Through step 16 to step 17 you can

use different render passes to make an
evenbetterresult ofyourcompositing!
The viewport now looks like this, and we

will take a look at how the render passes
are going tolooklikebased onthis. ↓
The viewport now looks like this.

(Skippable) Render Passes

16 In Cinema 4D, turn on ‘Render Settings’ (or press ‘Ctrl’ + ‘B’).
Go to ‘Octane Renderer’ (on the left) → ‘Render AOV group’, and then click ‘Enable’
under the ‘Render AOV group’ to enable render passes.
Then you can set the location for saving the different render passes, and the format is
recommended to be EXR or PSD.
Go to https://youtu.be/jBO7ywXpxCY to learn the details about render passes.
2. Go here
3. Click here
4. Set the file location
1. Go here
These 4 are often used

(Skippable) Material ID
17.1 The Material ID pass will give each of the different materials its own color, so that you
can easily pick out the object by its material from the composited image.
(Skippable) Object Layer Color

17.2 By using the ‘Octane Object’ tag, you can mark any specific object you want and have
it rendered in a certain color.
The Object Layer Color pass will render all the objects with an ‘octane object’ tag in the
color you choose for them respectively, so that you can easilty pick out the object you
want from the composited image.
Add an ‘Octane Object’ tag

Set the color you want

(Skippable) Z- depth
17.3 The Z-depth pass basically
gives you the information
of how close the objects are You can try different values on this setting
to the camera, and is often
used for effects like fogs.
(Skippable) AO
17.4 The AO (ambient occlusion)
pass is used to indicate the
amount of self-occlusion on
the surface, and it can simply
emphasize the volume of the You can try different values on this setting
objects.
Step by Step -
Affect Effect
The workflow of Compositing in AE is different from that of C4D.
There is no 3D modeling process, which makes it more suitable
for 2D scenes: for example (1) adding objects to the scene that
do not deform as the camera moves (there is no 3D modeling).
(2) Objects with insignificant three-dimensional depth or small
objects themselves, such as particles moving in space, light
beams, etc.
In addition, due to the speed of AE, AE is also very suitable for
simple removal and replacement of elements in the video. This
section will focus on two methods of performing removal.
IMAGE(S) OF YOUR DEMO PROJECT

IMAGE(S) OF YOUR DEMO PROJECT

AFTER EFFECT COMPOSITING: ELIMINATE OBJECT
A1 https://www.youtube.com/watch?v=U7Ch6zvp2m8
https://www.youtube.com/watch?v=nYoGBwdlVxw(6:00 Start)
Drag footage into composition and the timeline and grab the pen tool (Press G). The
mask does not need to match perfectly.
STAGE 1 PHOTO OR SCREENCAPTURE
A2 Change the mask to ‘Subtract’, then make key frame for the mask to make sure it cover
the object all the time. Open ‘Content-Aware Fill’ in windows, select object, then click
‘Generate Fill Layer’.
After the process, change the color setting and other thing to make the compositing
part looks more realistic. For example, add ‘Match Grain’.

A3 Click ‘Create Reference Frame’ then you can use photoshop to make a reference layer
beneath it.
https://www.youtube.com/watch?v=nYoGBwdlVxw (13:15)
A4 ‘Rotobrush’ is also usefull. It can help cut out the object you don’t want.
https://www.youtube.com/watch?v=Xv_p_fzudM4
This tutorial teaches how to select and track the objects, frame by frame, isolating the
subject automatically.
https://www.youtube.com/watch?v=SutrQtQT7FY
It’s a nice tutorial for removing object.

AFTER EFFECT COMPOSITING: ADD OBJECT
B1 Set up the file: double click to open After Effect. Press ‘Ctrl’ + ‘N’ to create new
composition the parameter as follow. If you cannot see the ‘Project File’ panel, go to
standard view. Also drag your video material to ‘Project File’.
B2 Drag the footage from ‘Project’ panel to the composition you just created.
If you choose to import PNG sequence footage you will need to check the frame rate.
B3 Find 3D Camera Tracker inside research box. It will calculate for several minutes
depending on how long the video is. If the footage have a ground it could have depth
for the video. Create a Null camera the the software can record the movement of the
camera.
STAGE ## PHOTO OR SCREENCAPTURE
B4-A The next thing is to choose the target you want to use.
Right click on the choosen mark to create a Null camera, then the the software can
record the movement of the camera.

B4-A1 Remember to use the point near the place you want to put your object to make
the null camera.
B4-A2 Make the object 3D by click the box.

Copy the position of the Track point to the object you have.

B4-A3 If you want to add more things, go back to the footage’s effect panel and click ‘3D
Camera Tracker’ the colorful target will show up again, you can create another
null object.
B4-A4 Add more things you want and don’t fogot to change the ‘Curves’ and ‘Resolutions’
etc. to make the footage looks more realistic.
https://www.youtube.com/watch?v=ax_wrRzDRUM

B4-B1 Also you can directly create a solid camera epecisally if your footage has depth.
The video we provide do not have a high resolution.
Watch this video to get more infromation:
https://www.youtube.com/watch?v=fKwTdWAxfg8&t=8s
B4-B1 Right click ‘Pre-Compose’, then add a ‘Track Solid1 Compo1’, then double click
to open it.

B4-B2 Delete the original green solid box and add your own footage.
B4-B4 Add post effect like motion bulr.
https://www.youtube.com/watch?v=-ZJKoNj4WpQ

FINAL OUTPUT
A clip of compositing videos from a street view and a drone shot.
https://youtu.be/KjEtcA9F0O0
EXPLAINER VIDEO
A tutorial video introducing the workflow of setting up scenes in
Cinema 4D, mapping and compositing and rendering out the final
output.
https://youtu.be/gqBzH0GGsyA

(Reference) VFX Compositing From The Phygital Laboratory - User Manual I

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

(Reference) VFX Compositing From The Phygital Laboratory - User Manual I

Uploaded by

Copyright:

Available Formats

266 TH E PH YG ITA L L A B O RA TO RY 2022

Visual eﬀects is the process by which imagery is created or

Aerial View VFX Compositing

Academy Award Winner

BAFTA Award Winner

Satellite Award Winner

VES Award Winner

“Gravity is a hybrid: it’s partly live-action, but is similar to [an

Green screen scene being composited into the

In this case study, the content creator, StudioBinder, walked

Imagine you wanted to create an explosion in an action movie. The

CASE STUDY IMAGE(S)

LIST OF REQUIRED HARDWARE

LIST OF REQUIRED SOFTWARE

With hand-held stabilizer

PHYSICAL SETUP DRAWING/DIAGRAM

1. INSTRUCTION ON TITLE COLORS

RED TITLES: Steps that are important and recommended to

2. LIST OF REQUIRED SOFTWARE

Cinema 4D R23 or newer (R25 in this case)

3. LIST OF TEMPLATE FILES

Image Sequence A folder including image ﬁles

Through step 01 to step 09, you will

Relocate the File Path

(Skippable) Import Still Image

Go to https://youtu.be/C0976dRXG_A, from 2:35 to 14:40 to learn how to use camera

*The camera calibrator is a diﬀerent workﬂow with camera tracking.

Projection Mapping - Create A Material

Press ‘Shift’ + ‘F2’ to open material manager to see the materials

Projection Mapping - Projection

Go to https://youtu.be/C0976dRXG_A, from 16:00 to 19:40 learn more details.

Click the material tag

Create an object, e.g. a cube, and

2. Click this small button and

Look at the right camera

The projection mapping should

Blockouts & Modeling

Go to https://youtu.be/C0976dRXG_A, from 16:00 to 19:40 to see how to set up the

Model the basic geometries and

*Increase the number of subdivision if the material looks distorted.

Shadow Catcher Material

Go to https://youtu.be/OeUgmej5rqE to learn how to use the shadow catcher.

(Skippable) Bake the Projection

The new UVW tag

Click here This one is changed to

Through step 10 to step 15 you can

Click here in C4D R25, or press ‘Shift’ + ‘F8’,

Match the Lightings

Rotate the direction

Balance the Contrast

Go to https://youtu.be/etOkgjrTfa8 to learn more about how to create and HDRI map.

Click here to choose

Click here to open

Tick the checkbox

‘Through step 16 to step 17 you can

The viewport now looks like this, and we

The viewport now looks like this.

(Skippable) Render Passes

4. Set the ﬁle location

These 4 are often used

(Skippable) Object Layer Color

Add an ‘Octane Object’ tag

Set the color you want