You are on page 1of 39

Creating a Virtual Reality Tribute to

Vincent van Gogh

Overview
-

Inspiration

-

Creative Approach

-

Technical Workflow

-

Tools

-

Learnings

The trailer for the experience can be watched here:
https://www.youtube.com/watch?v=jBOL5yakREA

• recording gear VR videos is tricky, I used a paid app called “Recordable”

Inspiration

Eureka!

• no eureka moment, the idea was an evolution

• original plan was for a film, fictional painter inspired by van gogh, went into the woods and battled a mythical bear

• I usually know when I have an idea I want to pursue when it keeps popping back into my head.

• stumbled upon an artist named Alexa Meade that made me believe an idea like this could work

• doesn’t require toon shading techniques, just carefully painted objects and characters

Le Café de Nuit (The Night Café)
1888

• I selected The Night Cafe because it was perfect for the technical requirements.

• It is an interior with a limited amount of space for movement.

many classic elements of VG style including radiating lights, skewed perspective, unusual combination of colors

• interiors are preferable because they are limited — no looking into infinity

Selecting the paintings:

• not all the paintings were from van gogh

• looking for objects to fill the spaces that needed to be imagined

• colors, shapes, and theme all informed painting selection

• mapping out the things I wanted to achieve with the demo

• initially the experience was going to have a lot of interactivity

• ultimately decided to focus on the visual experience due to time constraints / deadline

• I think the focus on atmosphere over interaction worked in the project’s favor for this experience

• although interaction could definitely add immersion as well

Research

=?

• What kind of lamp is this?

• What is an arc lamp? Is this an oil lamp?

• there are a lot of oil lamps.

• I also learned other interesting facts.

• Thomas Crapper popularized the flush toilet in the late 1800’s

Research = Authenticity

• researching history and visual references creates a more authentic world

• this charming place is called “Bandits Roost Alley” and inspired what you see through the window in the back

Pre-Production

• laying out the floor plan was very important

• helped to figure out where people could stand and what viewpoint they would have

• here is the added section compared with the original painting

• trying to complement van gogh’s color palette was a challenge

• I wanted to create a shift in tone in the added room but it still needed to feel like a cohesive whole

• I wanted it to feel more serene in that added room. you should feel a sense of ease as you go from the clashing colors of
the main room to this area

• here’s the final look in the experience

Asset Production Workflow

Design Rules

-

Everything should look unique

-

Things should blend into each
other

-

Each object and character should
convey mood

1) Everything should have a unique look. Things shouldn’t feel duplicated.

2) All the objects and characters should blend into each other as if brush strokes are intertwining.

3) On a more abstract level, everything should work towards a particular mood of being lost in a moment in time

Software Used
Modeling base mesh, UV layouts, rigging, animation
Sculpting details, painting textures
Game engine
Sketching, texture clean-up
Sculpting characters

Sketching

• the first step is to sketch out the object

• it is important to understand the form of what you will be modeling

Modeling

-

low polycount very important

-

total vertices in scene under 100k

• it was important to give each of these props a sense of flow

• first I constructed them with rigid forms

• keeping a low poly count is a consideration for every object considering this is a gear VR game

Modeling

-

low polycount very important

-

total vertices in scene under 100k

• I then used mudbox to warp the form to fit the painted style

Modeling

• characters had a higher poly count so I had to use them sparingly

Texturing

-

Handpainted in Mudbox

-

No overlapping UVs

• textures were all hand painted in Mudbox using a wacom tablet

• UV layout was very important, making sure there were no overlapping UVs so every surface area could be painted uniquely

Texturing

• characters typically took longer to paint because they incorporated more details

Samples of the modeling process can be seen here:
https://www.youtube.com/watch?v=jD-uM-Z6SN0

• texture projection wouldn’t work because of the surfaces you don’t see from the painting

• a lot of trial and error and making creative choices about how to best render non dynamic style lighting in a painterly style

Rigging
RapidRig = AWESOME!

• rigging is often tedious

• I discovered RapidRig on this project

• reduced rigging time from 6+ hours to 1

• caveat: you still have to paint skin weights

Animation
-

Kinect mocap is cool but low
quality

-

Hand keyframing is slow but worth
it

-

Fewer animations overall

-

Animation adds life to a scene

• After doing testing with 2 kinects to get mocap data I discovered the animations were unusable

• feet drift too much

• automated filters to remove noise kill the life in the animation

• had to keyframe it even if it meant less animation

• animation helped to give life to the scene

Particle Tests

• I did some very rough tests using particles to draw the color instead of shaders (wavy, flowing look)

• would likely not have worked on gear VR

• it was very tricky to achieve the details I wanted

• messy and would have taken much longer to develop

Specular Bumped Shading

• next I tested Specular Bumped shaders that incorporated normal maps

• normal maps don’t look as good in VR because they subtle lighting differences you would see in each eye are not accurate

• this style made it feel more like you were walking around in a painted box than inside the painting

Flat Shading

• Finally I settled on a flat shader

• looked the best because the colors were vibrant and matched the painting’s color palette exactly

• very good for performance (no normal maps, no per pixel lighting)

Controls
-

Simplified controls

-

Oculus SDK character controller
was 90% there

-

adding crouching for more control
over viewpoint

• after going back and forth on controls I decided simpler is better.

• not everyone has a gamepad for Gear VR

• even my gamepad wasn’t working properly (cost $60 bucks!)

•the SDK controller had most of what I needed but I had to modify it slightly, it was having issues using tap and hold to
move forward, built in rotation had to be disabled

• swiping down to crouch gave slightly more freedom to the player so they could see the environment from more vantage
points

Optimizations
-

Texture atlasing

-

Mesh batching

-

Texture/Audio compression

-

Reducing particle counts

-

Triggers to manually hide/show
objects (homebrew occlusion
culling)

• I used scripting to manually handle some optimizations

• particle effects were one of the biggest performance hits because they use alpha cutout shaders

• manually triggering hide/show on these helped with frame rate issues

Learnings

-

Characters in VR are really cool

-

Particles are awesome (obviously)

-

Movement is tricky

-

The Note 4 is pretty capable

-

Stylized worlds can still be
immersive

• as soon as I added characters to the environment that became my favorite thing to go check out

• there’s something about a moving, character that seems to have some life

• particles are really popular in VR. the blooming particle lamps are usually the first thing people comment about when they
try it.

• Movement is very tricky and I went through several iterations on control, speed, acceleration but settled on the most subtle
(slow, no acceleration, simple controls, strictly head rotation turning).

• you might isolate select players because they can’t handle any VR movement but it is a trade off.

• the Note 4 handled this better than I expected, it was originally going to be a DK2 demo

• stylized worlds without any shadows or per pixel lighting can still create an immersive experience that allows your mind to
day dream

Thank you!