UNREAL ENGINE 4 OPEN WORLD

TECH DEMO SECRETS

THE ART AND TECH BEHIND EPIC’S

GDC 2015 OPEN WORLD UE4 DEMO

Jack Porter
Engine Development and Support Lead
Epic Games Korea
 Topics
• Demo

• Creating Photorealistic Open Worlds

– Asset capture
– Large scale terrain lighting improvements
– Landscape setup
– Foliage and Grass and procedural placement tools
– Fauna/AI
– Target spec and managing performance
CAPTURING ASSETS IN THE FIELD
REFERENCE PHOTOGRAPHY AND PHOTOGRAMMETRY
 Biome
• Themed for Isle of Skye, Scotland

Striking contrast between rock and plant life.

 Logistics
• We decided on:
– 2 person team in NZ shooting for 7 days
• 12 hours of Daylight
• Flora in bloom
• Sunny
– 2 person team in EGUK ongoing
• Miserable weather but..
• Injects authentic UK assets
Photogrammetry

Reconstructing a 3D asset from photographs

 Gear
• 3 Canon 5D MKIII
• Sigma 8mm f/3.5
• Canon 24-70mm f/2.8L
• Color Checker
• Roundshot VR Drive
• Monopods
• Telescopic Pole
• Remote Live View
• Custom gray and chrome
balls rig

Gear
Shoot Process

Lighting Reference
Shoot Process

Custom bracketed panoramic HDR

Shoot Process

Get full photographic coverage

 Processing
• Captured 1.06 TB of pictures in 7 days
– 28,517 photos
– 248 total assets

• Massive amounts of data to be sent across the network for each

asset.
– Between 7GB-45GB per Asset
 Photoscan Reconstruction
• Photoscan textured mesh

7-20 million polys, 16K texture

 Process
• But asset has baked in lighting
– Not desirable for game assets
Process

Direct lighting
Process

Indirect lighting
Process

Dark shadow areas

 Delighting
• We recreate the lighting at the time of the shoot

Using a scripted semi-automated process

 Delighting
• Remove lighting from the photogrammetry texture

Resulting texture is called diffuse albedo

Final Results
 Assets now available for free!

These assets are

now available for
free on the Unreal
Engine 4
Marketplace!
LIGHTING
 Lighting

• Fully dynamic lighting

– Huge outdoor world
– Time of day changes
– Instant modifications (no build times)
– Large view range
 Lighting
• 3 main components of outdoor lighting we had to solve

Lit directly by Sun

Bounce lighting
from Sun
(Global Illumination)
Lit by Sky
 Sun Direct Lighting
• Cascaded Shadow Maps (CSM) near camera
– Supports foliage animation
– Large cost from triangles

CSM
 Sun Direct Lighting
• Ray Traced Distance Field shadows (RTDF)
– 3x more efficient for huge view range
– Maintains detail at all distances

RTDF
Shadows

CSM
 Sun direct lighting
• A Distance Field stores distance to the nearest surface at every point

Mesh Distance Field

 Sun direct lighting
Triangle scene Mesh Distance Field scene
 Sun direct lighting
• Ray intersection skips through empty space based on distance to surface!
– If the ray intersects, light is shadowed

Mesh Distance Field Ray Trace to light

 Sun direct lighting
• Bonus: Approximate cone intersection for free!
– Area shadows

Mesh Distance Field Ray Trace to light Cone intersection

Sun bounce lighting (New in 4.8)
Sky lighting only Heightfield GI
 Sky lighting
• HDR cubemap preprocessed for different roughness levels
 Sky lighting
• Shadowing
– Screen Space Ambient Occlusion (SSAO) for creases / distant
– Distance Field Ambient Occlusion (DFAO) for medium scale

SSAO only With DFAO

 Sky lighting
• Distance Field Ambient Occlusion
– Traces cones against mesh distance fields to determine sky visibility
– Computed with world space occluders, no screen space artifacts!
 Lighting setup
All lighting setups are controlled from UE4 Blueprints
depending on the demo mode:

• Per-scene lighting during Kite cinematic

• Attract mode
• Free camera mode with Time-of-Day changes
 Lighting setup
Basic lighting setup contains:

• Directional component (Sun)

• Skylight component with ambient cubemap captured
from asset shoot
• Light Function used for cloud shadows on the ground
• Height fog
LANDSCAPE & FOLIAGE
Landscape

16 x16 km, 1m resolution

 Landscape

Continuous LOD system with streaming

 Landscape
• Heightmap and layer masks generated in World Machine

• Divided into tiles and imported using World Composition

• Blended textures based on

– layer masks
– also height/surface direction

Distant landscape tiles are unloaded and replaced with Streaming Level
LOD Proxy
Landscape Material

Weight maps
Landscape Material

Result: textured, lit material view

Landscape Material

Raw weight map

Landscape Material

Simple blend
Landscape Material

Blend using ‘height map’

Landscape Material
 UE4 Foliage Systems
Foliage Grass

• Trees, rocks… • Grass, flowers, pebbles…

• Collision • No collision
• Manual/procedural placement • Material driven placement
 UE4 Foliage Systems
New In UE 4.7:
• Foliage Painting Tool performance increased using “Hierarchical Instanced
Static Mesh”
– Render over a million instances
– Organize meshes into BSP-like tree
– Per-pixel LOD transitions

New In UE 4.8:
• Procedural Foliage placement tool (Experimental)
• Landscape Grass
• New UI for Foliage Painting Tool
Foliage Painting Tool
Foliage Painting Tool
 Foliage LOD
LOD levels are specified in the
Static Mesh editor

• We typically used 3 or 4
LODs

• eg Tree
– 115,000 triangles
– 13,000 triangles
– 2,500 triangles
– 32 triangles (billboards)
 Foliage LOD

LOD polys: 115,000 13,000 2,500 32

 Foliage Shadowing
• Use pixel depth offset on LODs to improve shadows
Two sided shading model
 Two sided shading model
• Models light transmission through
thin surfaces

• Used for leaves and kite fabric

 Procedural Foliage
Goals
• Natural looking
– Clustering
– Self Thinning
– Growth Curves
• Fast iteration
• Artist intervention
 Simulation
Create a ‘ecosystem’
containing species

Add a few instances

of each species,
based on density
settings
 Simulation
Each step we age
the plants and
spread seeds
 Simulation
System can also
place rocks, bushes
etc.
 Simulation
Use species rules to
decide overlaps
• Relative Size
• Shade
• Altitude
• Slope
Simulation
 Simulation
Final result!
 Tiles
Natural simulation for
entire world too
expensive

Compute 2D tiles and

apply them

Use rules to cull trees

 Tiles
Volumes act as masks
 Procedural Foliage - setup
• Create Procedural Foliage Spawner assets to control foliage creation
– Specifies a list of Foliage Types to generate
 Procedural Foliage - setup
• Create a ProceduralFoliageVolume and set
the Spawner type
 Procedural Foliage - setup
• Press the Resimulate button to run the simulation
and generate foliage
 Painting
Deterministic
simulation, allows for
easy touch ups

After simulation, artists

can edit result like
ordinary foliage
Demo Foliage
Demo Foliage - Stats
Rocks 193,000

Trees 226,000

Bushes 1,170,000

Flowers 6,358,000

Grass 7,879,000
Excluding grass system!
 Landscape Grass
• New in UE4 version 4.8

• Grass placed on Landscape

only

• No collision

• Suitable for
– grass, flowers, pebbles etc

• Placed automatically around

the camera using a density
mask
 Landscape Grass setup 1
Create Landscape Grass Type assets to
describe the grass meshes you want
placed on your landscape

Each Landscape Grass Type can be placed

in different areas of the landscape, but
can also overlap

Each grass type is placed according to a

layer mask
 Landscape Grass setup 2
Each Landscape Grass Type can specify multiple
meshes called “Grass Varieties”

• Each variety can have its own density, placement and

LOD settings

• After considering the mask which affects the entire

Grass Type, individual variety meshes are placed
according to their densities
 Landscape Grass setup 3
Create a LandscapeGrassOutput node in your Landscape’s material and
name and assign your grass types.

Each grass type adds a new

node input.

The inputs are used to create

the mask for the grass type,
specifying where the grass will
be placed
 Landscape Grass - runtime
Using weight maps
directly did not work well

Idea: Use the Material

graph to define grass
density!

Advantages
• perfect fade out to
texturing
• rules for culling
instances in material
Landscape Grass fade out
FAUNA
 Fauna
• Our world is very pretty.. but static

We wanted to add some life!

 AI
• Find path from centre of herd away from drone
Navigation Mesh
 Animation
• Blended root motion
– AI gives a goal
– State machine finds best anim
Animation
Animation

Optimizations: 300 deer

 Spawning
– Divide world into cells
– Rules determine best locations in each cell
TARGET SPEC & PERFORMANCE
 Target Spec
For this demo we targeted a very high-end PC

– Intel Core i7 5820K

– 32GB RAM
– Nvidia GTX 980 and later Titan X
 Target Spec
However, we could have targeted a lower spec such as a lower-end graphics
card or PS4.

– Demo is limited by GPU performance

– Demo could easily be scaled by scaling content

• Triangle counts
• Texture sizes

– You must test and adjust content to achieve performance on your targets on
your target device
 Performance
• During development we gathered daily performance stats of our
cinematic on our target hardware – mainly GPU and Rendering Thread
(drawcall/driver) CPU time
 Optimizing Scenes
Main problem was GPU
time caused by huge
triangle counts

• Up to 20M triangles in
some scenes!

• Reduced triangles in
LODs

• Adjust LOD view

distances

• Adjust grass LOD

parameters
CONCLUSION
 Conclusion
• Photorealistic Open Worlds are possible with Unreal Engine 4!

• The captured assets are currently available on UE4 Marketplace.

• The full demo will be available on UE4 Marketplace after UE 4.8

release
 Q&A
• Thank you!