3D STEREO RENDERING

WHITEPAPER
NEW PROCESSING & PERCEPTION CHALLENGES
 
 
 
 
 
 
 
 
Disclaimer.  Statements  and  opinions  expressed  in  the  speeches,  commentary, 
articles and communications herein are those of the author(s) and not necessarily 
those of the 3D@Home Consortium or its members. 3D@Home and its members 
disclaim  any  responsibility  or  liability  for  such  material  and  do  not  guarantee, 
warrant, or endorse any products or services advertised in this publication, nor do 
they guarantee any claim made by the manufacturer of such products or services. 
 
 
 
 













3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010 | 2
CONTENTS
INTRODUCTION ............................................................................................................................................................................................. 3
“3D STEREO RENDERING” – DEFINITION AND CONTEXT ................................................................................................................................ 4
DEFINITION......................................................................................................................................................................................................................... 4
CONTEXT ............................................................................................................................................................................................................................ 4
TECHNICAL CHALLENGES ............................................................................................................................................................................... 6
OVERVIEW.......................................................................................................................................................................................................................... 6
INPUT DATA MANAGEMENT .................................................................................................................................................................................................. 7
SIGNAL ENRICHMENT ........................................................................................................................................................................................................... 8
3D INSERTION..................................................................................................................................................................................................................... 8
OUTPUT FORMATTING ....................................................................................................................................................................................................... 10
PERCEPTION CHALLENGES ........................................................................................................................................................................... 12
OVERVIEW........................................................................................................................................................................................................................ 12
OBSERVATION CONDITIONS ................................................................................................................................................................................................ 12
HUMAN FACTORS .............................................................................................................................................................................................................. 13
CONCLUSION .............................................................................................................................................................................................. 15
REFERENCES ................................................................................................................................................................................................ 16
ACKNOWLEDGMENT ................................................................................................................................................................................... 16
LIST OF FIGURES .......................................................................................................................................................................................... 16


ABSTRACT— The advent of 3D to the home raises a number of challenges beyond the basic display of an available stereo
pair. A number of them are technical challenges, for example the adaptation of various 3D content types to different
display technologies. Besides these, perception challenges relate to the quality of the experience, fatigue, eye strain, or
variability of perception, and involve observation conditions and observers.
This paper presents a basic tutorial on “3D Stereo Rendering” technology, a consumer-side constituent of the 3D
delivery chain. As a shorthand way of saying “Adapting decoded 3D content to each target 3D display technology —
taking into account Video Quality and Human Factors”, “3D Stereo Rendering” brings together perceptual understanding
and know-how, algorithms and video processing techniques.
3D stereo rendering can be implemented as a set-top, gateway, or software-processing block, getting its input signals
from 3D decoding unit(s) and providing output signals to various 3D display technologies. A number of processing modules
need to be developed and assembled to perform 3D rendering. These include disparity or depth map extraction, view
interpolation, subtitle and graphic insertion, color and disparity processing, resampling, etc. While some modules depend
mainly on the technical environment – as display characteristics and signal formats – some others will depend on observer
characteristics and observation conditions. This is a new and fundamental aspect in the design of home entertainment
products.
The aim of this paper is to present and discuss the topic of 3D stereo rendering, and the related technical and
perception challenges. After a general introduction and some definition, a modular structure is presented, first
addressing technical challenges and introducing processing modules related to quality-of-experience enhancement. A
following section introduces the needs for perception-related processing and, finally, elements are given to address these
perception challenges thanks to automatic or user-adjustable 3D rendering.
KEYWORDS— 3D Displays, Human factors, Stereo vision, TV, Video processing










3 | 3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010

INTRODUCTION
N JULY 2005, a joint venture between major studios, the DCI -Digital Cinema Initiative, published the first version of its
“Digital Cinema System Specification” [1] document. The purpose of this document was to define the digital representation
playback and display of feature films in movie theaters. From this point, the industry could start to develop and produce
projection systems as well as content for digital cinema. This was the start of a new cinema content production and
consumption workflow, which now runs every day: digital cinema. On July 11, 2007, the DCI published the “Stereoscopic
Digital Cinema Addendum” [2], a one-page complement fixing the use rules for 3D digital cinema representation and display.
Defining only a few compatibility points and which image should come first, this document set up the rules. As the most
important consequence, it broke the causality dilemma for 3D content and 3D display of stereoscopic movies, enabling the start
of a circle: the content creation industry could go 3D with well defined rules and formats (Figure 1).


Now, only a few years later, as more and more 3D releases are planned, we see the other half of the circle, i.e., the
consumer industry, taking over. Blu-ray has already specified stereo compression schemes to deliver 2D and stereo content
simultaneously. A large majority of manufacturers are announcing 3DTV products for the coming months and, in parallel,
standardization organizations like HDMI, SMPTE, MPEG and DVB are preparing standardization of future 3DTV formats to allow
deployment in homes. The 3D@Home consortium also contributes.

The advent of 3D to the home raises a number of challenges beyond the basic display of an available stereo pair. First, the
technical environment is not yet established, with various formats and display types. Secondly, the observation conditions are
quite different from those of the movie screen, with smaller audiences and relatively short distances between the viewers and
the screen. On one side, we see technical challenges, such as the adaptation of various 3D content types to different display
technologies. On the other side, perception challenges relate to the quality of the viewing experience, involving observation
conditions and observers.

I

Figure 1 – 3D Virtuous Circle Since 2007


1. More and more 3D releases
2. 3D Standards
3.
3D Delivery and 3D Displays
and more…
2007
2008
2009
2010:
19 planned













3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010 | 4
This paper will first introduce the 3D stereo rendering topic, and then will describe the related technical and perception
challenges. Finally, the paper presents some suggestions for addressing these challenges.



“3D STEREO RENDERING” – DEFINITION AND CONTEXT
DEFINITION
“3D STEREO RENDERING” is used in this paper as shorthand for “Adapting decoded 3D content to each target 3D display technology
— taking into account video Quality and human factors.”
The goal is to ensure the best possible 3D experience for a given observer on a targeted display.

As such, this subject matter covers processing elements in the video chain applied on decoded 3D video content and
providing signals to 3D displays (Figure 2).


Addressing this topic involves:
• Perceptual understanding and know-how
• Algorithms and video-processing techniques



CONTEXT
3D STEREO RENDERING can be implemented as a set-top, gateway, or software-processing block to perform the task of
“adapting decoded 3D content to each target 3D display technology.” As defined above, it is getting its input signals from 3D
decoder(s) and providing output signals to various 3D display technologies.

While the principles and algorithms we will describe also can apply to 3D cinema or 3DTV content production, this paper will
focus on the consumer 3DTV/3D video chain (Figure 3). In this case, 3D stereo rendering can be part of the same equipment or
processing unit as 3D decoding.

3D rendering requires various processing blocks to be developed and assembled, with parameters depending on:
 Display characteristics and viewing conditions
 Observers’ characteristics and commands


Figure 2 – Rendering Module: an Element of the 3D Video Chain











5 | 3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010

As described in next chapters, 3D rendering processing blocks will include 3D signal-processing operations like resampling,
disparity or depth-map extraction, view interpolation, subtitle or graphic insertion, and color and disparity processing.




3D rendering is an extension of the classic 2DTV “signal processing for displays,” such as de-interlacing, spatial scaling or
simply color management. In this 3D technical area, we find identical concerns of display dependence, the need to take into
account human perception in algorithms optimization, and observer control.

In 3D rendering, however, challenges appear more complex and sophisticated than in the 2D case, and include both technical
challenges and perception challenges.

 Technical challenges are linked to signal input, processing and adaptation to displays.
 Perception challenges relate to the viewers and to observation conditions.

The following sections will describe more in detail these 3D challenges, and will give examples of new components
considered to improve the 3D experience.


Figure 3 – Rendering Module: Application Context

Embedded
3D Rendering
engine
Set-tops / Gateways
3D Rendering
Observer Characteristics and Commands
3D Signal & Metadata
Display Characteristics /
Viewing Conditions













3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010 | 6
To evaluate performances, a new range of 3D quality of experience tools has to be developed, supplementing the 2D quality
tools available. (For example, see an article entitled, “VQEG Validation and ITU Standardization of Objective Perceptual Video
Quality Metrics” [4].)


TECHNICAL CHALLENGES
OVERVIEW
A number of modules together compose the 3D stereo rendering chain. Examples of the technical processing aspects
include:
 Input data management module: Converts from the decoded signal output to the 3D stereo
rendering-specific formats.
 Signal enrichment module: Creates complementary signals of use in 3D stereo rendering, such as
depth maps
1
or disparity maps
2
 3D Insertion module: Responsible for the insertion of visual elements, such as subtitles, logos or
score boxes.
.
 Output formatting module: Prepares signal in correspondence with the actual display
characteristics.

All of these modules require complex and efficient algorithms to be developed as detailed below. A substantial part of the
challenge is to run them on restricted resource platforms, which, nevertheless, satisfy the functional needs. Figure 4
represents a tentative processing structure combining these modules and the perception-management module described in the
next section.

1
Depth map: content related, pixel by pixel, distance information between scene point and projection point (analogous to Z-Buffer
information).
2
Disparity map: in-plane vectors between corresponding pixels of a left and right image (analogous to an optical flow map but related to a
stereo image pair).










7 | 3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010


Figure 5 – New 3D features: Supported by
standards

INPUT DATA MANAGEMENT
In the current state, no standard is established for the
representation of transmitted 3DTV stereoscopic content. It is the
purpose of SMPTE and DVB to propose such a format. In parallel,
HDMI also will publish a 3D broadcast format specification to ensure
interoperability of 3D-capable equipment. Both standards are
expected very soon.

In the meantime, input data processing must be provided to
adapt the video format, resample spatially or temporally, or to
reconstruct stereo views. When standards are published, the
complexity of this module may decrease. Still, there may be
several different profiles that require signal adaptation.
Furthermore, 2D compatibility of the signal and legacy equipment
will require an adaptation of the content to the real capabilities of
the display.

This module also may include the correction of imperfect image pairs, for example random compression blocks, which create
severe visual annoyances in the form of 3D structures.


Figure 4 – Rendering Module: Processing Structure













3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010 | 8

Figure 6 - 3D Graphics management
SIGNAL ENRICHMENT
If it is not available in the decoded data, complementary information, such as depth or disparity maps, must be created for
the subsequent processing tasks. Such signals can be estimated from the stereo pairs, sometimes with the support of
metadata. This step is quite challenging, as algorithms often are elaborate and complex, principally in the steps of stereo
matching and occlusion management [5].


3D INSERTION
The term “3D insertion” is used here as an extension of the 2D video
term, “overlay.”

In 2D video, a number of visual elements commonly complement the
programs. Examples of these are subtitles, menus, score boxes or
logos. Some, such as subtitles or menus, are dependent on the
consumer choices – and thus need to be inserted locally – while score
boxes or logos can be inserted at the post-production stage. In all
cases, the problem is how to correctly insert these elements in the 3D
content to avoid visual annoyance when the inserted elements’ depth is
different from the scene depth and structure around it. The situation is
even worse when masking by foreground scene elements occurs.

For example, with subtitle insertion, the easiest option is to let subtitles be in the screen plane, which generates discomfort
due to occlusion by the content. Placing the insertion in front of any object in a sequence, in turn, may create highly crossed
disparities that are difficult to fuse. Using depth extracted from the stereo pairs, intermediate strategies can be developed,
such as depth-dependent subtitle placement or depth adjustment of the whole scene. (See Figure 7). Such solutions reduce
visual discomfort at the cost of depth extraction and depth-dependent processing.












9 | 3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010


Two examples are illustrated in this figure, both exploiting both a depth map.
1. Subtitle placement depending on scene depth
2. Scene depth translation to place the subtitle in the screen plane

These examples apply mainly in the situation in which content appears in front of the screen plane (and has
negative disparity).

The depth map can be extracted/computed at the post-production level, then transmitted and used by 3D
rendering. Or, in the case of equipment having more processing power, a depth map can be computed in a specific 3D
rendering module.

In the first example, the depth map is used to evaluate the foremost object distance for each image

of a sequence.
Then, applying a temporal filter on these distances
,
a smoothed distance


is computed, where the subtitle is inserted.
In this situation, much less effort is required of the viewer compared with the case in which the subtitle is placed in front of
any content element during a full sequence.
In the second example, the subtitle is fixed at a defined distance, for example the average or median distance

of content
elements during a full sequence. In that situation, low effort is required, on average, to converge from the content to the
subtitle, and vice-versa. Some occlusions happen, but they can be processed by blurring the subtitle portions masked by a



Figure 7 – Rendering module: Examples of using a depth map in subtitle insertion
(Note: The 3D processing cannot be rendered in this paper, as it requires a stereoscopic display to be seen.)


Subtitling adapted to the content. Depth
information generated from stereo content
is used to optimize subtitle rendering
Video processing (e.g. scene depth
translation backward ) is used to reduce
risk of visual discomfort
Depth map extracted from
stereo pairs













3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010 | 10
content element, for example. Such blurring will make the subtitle/content collision clear and understandable to the human
visual system, reducing annoyance. This blurring process is performed using the depth map compared with the subtitle
distance.

In conjunction with this local subtitle blur to increase comfort, it is desirable to place the subtitle in the screen plane in
order to lessen the accommodation vs. convergence conflict. To do this, the full content has to be translated in the depth
direction of a distance . Although a basic approach is to shift the left and right images of the stereo pair, one
relative to the other, a much more correct processing method is to re-interpolate the image pair taking into account the depth
map. With this depth information, the translation


can be precisely applied to each pixel. This transformation cannot be
modeled by a global image shift, and the use of the depth map results in a substantial quality increase of the depth-translated
scene.
As can be seen, 3D insertion requires sophisticated processing that exploits depth information and insertion strategies
depending on the scene structure and its evolution in the video.

OUTPUT FORMATTING
Essentially, the previous processing modules will provide full-resolution stereo pairs at a given frame rate. The goal of the
output formatting module is to convert this signal to the actual display connected, depending on its technical capabilities.

As of today, a number of technologies satisfy stereoscopic-TV needs, to one extent or another (see Table 1).

Anaglyph is a passive technology candidate, using color opponents for the left and right views (e.g. red/cyan). One
advantage of anaglyph technology is that it can be compatible with already-deployed 2DTV displays. However, this technology
suffers color/3D quality issues and can generate binocular rivalry, i.e. perception conflicts between the two views.

Polarization is another option, with micro-polarizers applied line by line on the screen and polarized glasses selecting the
polarized light for each eye. The main disadvantage is the loss of spatial resolution. No mainstream product using this
technology has been announced.



TABLE 1 – STEREOSCOPIC DISPLAY TECHNOLOGIES

Glasses type Display technology Pros/Cons +/-
Passive glasses
Anaglyph
2DTV displays deployed
Color/3D quality issues
Binocular conflicts
+
- -
- -
Polarized
Light & cheap glasses
3D-specific technology
No full resolution
No mainstream products
+
-
-
-
Metameric
View separation and colors
Full resolution
Available in projection
No consumer screen available
+
+
+
- -
Active glasses Sequential, with shutter-glasses
Full resolution
Possible flicker
Low light
Available in Q2’10
+
-
-
++










11 | 3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010


Metameric
3

color separation is a third category of passive stereo displays. This technology supports full spatial resolution
and is available in projection, but no consumer screen is available.
Sequential display of left and right images can be realized with LCD or plasma technologies, separated by synchronized
shutter glasses. Full resolution is available, with some flicker possible and low light level for the demonstrated solutions.
Consumer products will be on the market in 2010.

Depending on the technology, the “native” display image structure may take different formats, such as color mixed,
interlaced or temporal display of left and right views. On top of this, in most cases, the display can accept various input signals
with different forms: single HD, side-by-side, over-under, various interlacing – line-by-line or quincunx – and finally, full HD
resolution for each view. All of these options are converted internally to the native format of the display.

From the 3D rendering point of view, the display type can be revealed through manual set-up or by automatic, e.g. HDMI,
signaling. Optimally, the 3D rendering system then would know both the native and other accepted input formats. Depending
on various criteria (including complexity and quality), it will choose the best format to render.

Output formatting processing can include a number of elementary signal-processing functions, such as 2D filtering,
subsampling/oversampling, interlacing views, temporal processing with or without motion compensation, color processing, etc.
Another 3D-specific point is crosstalk before correction, when imperfect image-separation techniques in the display create
ghosts, shadows or double contours. This effect and its correction are strongly content-dependent. That is, the effect
increases with enhanced contrast and with increasing disparity values.

The complexity of these modules depends on design choices, on the availability of standardized formats, on the display
technology artifacts, and on the choice of whether to delegate the processing to the display – potentially compromising quality.


3
Metameric: using colors with different spectral distribution but identical color perception in average













3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010 | 12
PERCEPTION CHALLENGES
OVERVIEW
Analyzing viewers’ reports, some of which are enthusiastic and others alarming, one cannot assume that there is a correct 3D
perception for everyone for any of the current 3D display technologies. Fatigue, eye strain and the variability of perception
are important and they cannot be neglected when developing a 3D chain.

It is necessary to better understand how we really see the 3D effect and how stereoscopic rendering can correctly satisfy
perception. Here are a number of open questions:
• How can we improve the 3D experience globally?
• What is the relationship between display size/viewing conditions and the 3D experience?
• How can we make this experience more attractive and less painful?
• Which phenomena/parameters in 3D vision or in the 3D system are responsible for visual fatigue?
• Which element of the human vision could induce a good or a bad 3D experience?
• Are we all at the same level regarding the 3D experience?
And, to sum up:
• How may we to best translate/convert the 3D cinema experience to 3DTV screens?

Observation conditions are the first aspects to address. Viewing distances, field of view or other parameters are drastically
different in a consumer environment that they are in a movie theater. Accordingly, the content has to be adapted.

In addition, each viewer has a different physiology and vision history. He or she may have preferences regarding the way
stereoscopic images are presented, in correspondence with his or her own 3D perception.

OBSERVATION CONDITIONS
A widely discussed subject regarding comfort of 3D viewing is the vergence-accommodation
4
[6]
conflict. Vergence vs.
accommodation decoupling happens when displayed 3D objects are not portrayed in, or close to, the screen plane. The
decoupling often is cited as the main cause of visual discomfort and fatigue. The vergence-accommodation conflict is a well
known problem with 3D displays. It already has been described in scholarly articles .

Image shifting is a simple solution that usually is proposed. This solution adjusts horizontal disparity on the screen. It allows
reduction of the disparity between the stereo image pairs, placing objects in the screen plane, for instance. However, the 3D
scene and represented distances are affected. There is a risk of incoherence in depth rendering, creating the perception for
the viewer of distortion of space.

On the same subject of image distortion, problems may arise from optical aberrations of the acquisition camera systems
5

or
in subsequent processing. These issues may hinder a comfortable perception of the rendered scene if they do not fit with the
specific display conditions.
Very often, observation conditions, (as well as acquisition conditions) are described by simple geometric models, the
simplest one being the pinhole (stenopeic) model, or central projection commonly used, for instance, in CGI
6

4
Here, accommodation is the change of optical power to maintain focus on an object, while vergence is the simultaneous movement of both
eyes to adjust the gaze directions on a same object for binocular vision.
.
5
or by the CGI camera model.
6
CGI: Computer Generated Imagery.










13 | 3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010


However, to better represent reality, geometric models need to be enhanced to include more physical parameters
corresponding to human vision. The importance/impact of each parameter needs then must be evaluated to determine if it
should be part of a recommended/reference model.

Lighting is another aspect of observation conditions that must be understood and modeled appropriately. For example,
pupils contract in high brightness, thus increasing the depth of focus of the eye.

HUMAN FACTORS
Physiological and perceptive factors contributing to 3D perception are numerous. A basic one, often cited, is the inter-
ocular distance defining the stereoscopic epipolar base. It is commonly used as a parameter to transform the stereo pairs for
varying viewing distances or conditions in algorithms modeling the geometry of observation. However, 3D space perception
goes far beyond exploiting geometry. A successful model is the probabilistic combination of a variety of 3D cues, as shown in
[7]. Complementing binocular disparity, binocular and monocular 3D cues include accommodation, convergence, parallax,
perspective, occlusion between object, and relative and familiar size.



While the content creator may play with these cues and their interpretation, collision of monocular or binocular cues will
rapidly create inconsistencies and annoyances. Thus, it is of interest to understand how strong these cues are relative to each
other and compared to stereoscopic disparity. It is also of interest to evaluate the acceptance/annoyance in perception in case
of conflicts.

A number of effects also disturb stereoscopic viewers, among them:
• Cardboard effect: Stereoscopic distortion resulting in an unnatural depth perception, where objects appear flat,
as if the scene were divided into discrete depth planes.
• Puppet-theater effect: A miniaturization effect that makes people in the scene look like animated puppets.
• Sudden changes in scene depth caused by abrupt scene changes, requiring the re-adaptation of the observer’s
vision
• Rapid objects’ depth changes: Motion of objects causing rapid vergence changes, which the observer has to
follow.
• Appearing/disappearing border objects: An object at the border of the picture is not present on both stereo
pictures.
• Expanded depth of field: On some sequences, depth of focus is larger than in real life. The viewer may have a
sharp and double vision of normally blurred objects.


Figure 8 –- Some monocular depth cues
Familiar size
Depth blur
Perspective
Perspective
Relative size
Occlusions
Aerial
diffusion













3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010 | 14
All these effects, and more, create conflicts for the visual system and discomfort. Understanding and ranking the effects
and analyzing the cause of discomfort are the first steps. Then, video processing methods must be developed to better match
the condition of acceptance for each potential viewer.

It appears that observers perceive annoyance to varying degrees; effects that some viewers perceive as “strong” are almost
ignored by others. This is called “named observer variability” and it can be related to physiology, neurology and vision
alteration, deficiency or imperfection, making space/depth perception different from one observer to the next.

Observer variability related to space/depth perception needs to be studied, and processing must be adapted to each type of
vision/perception. For example, “The relationship between CA/C ratio and individual differences in dynamic accommodative
responses while viewing stereoscopic images” [8] describes inter-individual differences in terms of
accommodation/convergence and identifies two types of behavior in the general population. Other papers, such as “Visual
discomfort and visual fatigue of stereoscopic displays: a review” [9] provide a review of the causes of visual discomfort and
visual fatigue when watching stereoscopic displays and identify variability among observers.

On this topic, a proposed solution is to provide the viewer with control over the strength of the depth effect. This can be
realized by interpolation a number N of intermediate views between the two views of the original stereo pair. Then, selecting
a new stereo pair among these N views, the system can display the content with a reduced disparity, and consequently less
depth effect. Figure 9 illustrates how views can be interpolated from an original stereo pair and associated disparity map in an
STB when a new pair is selected to provide variable depth strength, ranging gradually from the full original 3D strength down to
2D (no depth).

This content adaptation can be adjusted by the viewer, depending on his/her preferences or tiredness state, using a
dedicated remote control push button, for example. This feedback/control from the viewer(s) toward 3D rendering is very
desirable. It is not feasible in a movie theater, but rather it could be used by a small group of viewers or an individual
watching 3DTV.












15 | 3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010


CONCLUSION
To have wide success, 3DTV requires acceptance by a large portion of the available audience. Technical and perception
challenges must be addressed, taking into account the home visualization equipment as well as observation conditions, and
giving options to the viewers to adapt stereoscopic content to satisfy their individual perception. Without such modules in the
3D chain, there is a significant risk that dissatisfied customers would complain of poor perceived quality, headaches or eye
pain, and potentially would reject 3DTV. Therefore, the objective of the industry should be to propose automatic or
interactive features that enable the visualization system to perform at its best, and to allow the consumer to enjoy
comfortable stereoscopy.

In order to reach that goal, we must develop adequate processing modules, allowing the user to take control over some
aspects of rendering. He or she will adjust 3D to his/her own state of perception or fatigue, and make the experience more
comfortable.







Figure 9 – User-adjustable 3D: from 2D to full 3D strength
leftview
right view
View
interpolation in STB
3D intensity adjustment
Stereo
Content
Dense disparity
map extraction
3D+
3D-
3D full
mode
2D mode
Stereocontent +disparitymapbroadcasted End-user selectionbetween3D full mode and 2D mode













3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010 | 16
REFERENCES
[1] Digital Cinema Initiatives LLC, “Digital Cinema System Specification”, Version 1.2, March 07, 2008 [latest version].
Available: http://www.dcimovies.com/
[2] Digital Cinema Initiatives LLC, “Stereoscopic Digital Cinema Addendum,” Version 1.0, July 11, 2007. Available:
http://www.dcimovies.com/
[3] G. de Haan, “Video Processing for Multimedia Systems,” ISBN: 90-9014015-. University Press Eindhoven, 2000.
[4] K. Brunnström, D. Hands, F. Speranza and A. Webster, “VQEG Validation and ITU Standardization of Objective Perceptual
Video Quality Metrics,” IEEE Signal Processing Magazine, May 2009, pp. 96-99.
[5] Q. Yang, L. Wang, R. Yang, H. Stewénius and D. Nistér, “Stereo Matching with Color-Weighted Correlation, Hierarchical
Belief Propagation and Occlusion Handling,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 31 N°3,
March 2009, pp. 492-504.
[6] D. M. Hoffman, A. R. Girshick, K. Akeley and M.S. Banks, “Vergence–accommodation conflicts hinder visual performance
and cause visual fatigue,” Journal of Vision, 8 (3), (2008), pp. 1-30.
[7] D. Kersten, P. Mamassian and A. L. Yuille, “Object perception as Bayesian inference”, Annual Review of Psychology 55,
(2004), pp. 271-304.
[8] T. Fukushima, M. Torii, K. Ukai, J.S. Wolffsohn, B.Gilmartin, “The relationship between CA/C ratio and individual
differences in dynamic accommodative responses while viewing stereoscopic images”, Journal of Vision, 9 (13):21, (2009),
pp. 1-13.
[9] M. Lambooij, W. Ijsselsteijn, “Visual discomfort and visual fatigue of stereoscopic displays: a review”, Journal of Imaging
Science and Technology, (2009), 53(3), pp. 030201.


ACKNOWLEDGMENT
This paper reflects work addressed by the “Content for 3D/3D Delivery to the Home” workgroup of Technicolor Research.
Many thanks to all workgroup members for their contributions.


LIST OF FIGURES
Figure 1. 3D Virtuous Circle Since 2007 ............................................................................................. 3
Figure 2. Rendering Module: an Element of the 3D Video Chain ................................................................ 4
Figure 3. Rendering Module: Application Context ................................................................................. 5
Figure 4. Rendering Module: Processing Structure ................................................................................ 7
Figure 5. New 3D Features: Supported by Standards .............................................................................. 7
Figure 6. 3D Graphics Management .................................................................................................. 8
Figure 7. Rendering Module: Examples of Using Depth Map in Subtitle Insertion ............................................ 9
Figure 8. Some Monocular Depth Cues .............................................................................................. 13
Figure 9. User Adjustable 3D: from 2D to Full 3D Strength ..................................................................... 15










17 | 3D STEREO RENDERING: NEW PROCESSING & PERCEPTION CHALLENGES | May 2010

AUTHORS
Laurent Blondé is graduate engineer of the Institut d’Optique - ParisTech (1985). Hired as a research engineer for
Thomson/Technicolor R&D, he participated in several R&D projects, including Infrared Image Synthesis, Special Effects and
Virtual Studio, Display processing, Anti-Camcorder and Color Management for Cinema applications. Supporting Technicolor
businesses, Blondé currently is a principal scientist. His research interests involve all domains of image processing for the
media industry, with physics and perception in mind.


Didier Doyen is graduate engineer of the Institut National des Sciences Appliquées in Rennes (1989). In 1990, he joined
Thomson/Technicolor, where he was involved in signal-processing activities. He drove projects aiming at improving the picture
quality of different display technologies, and more recently he has been involved in projects related to piracy prevention in the
digital cinema environment. This last activity required a study of the human color vision system to better understand its
consequences in term of color perception.


Thierry Borel is graduate engineer of the École Supérieure d’Électronique de l’Ouest in Angers (1986). He joined
Thomson/Technicolor in 1987 and has been involved in flat display technologies and addressing algorithms. In 1998, he was
appointed display lab manager, and he has conducted various projects aimed at developing emerging flat displays (DLP/LCoS
projectors, LCD, plasma and OLED) and the associated image processing methods. Since 2008, he has been manager of the
Signal Processing & Acquisition Lab at Thomson/Technicolor Research in Rennes. He also leads a research project related to
3DTV, addressing acquisition, distribution and rendering of 3D stereo and multi-view content. He is the coordinator of the FP6
Osiris European research project.

















TECHNICOLOR WORLDWIDE HEADQUARTERS
1, rue Jeanne d’Arc
92443 Issy-les-Moulineaux, France
Tél. : 33(0)1 41 86 50 00 - Fax : 99 (0) 1 41 86 56 15

www.technicolor.com
SALES CONTACT



© Copyright 2010 Technicolor. All rights reserved.
All tradenames referenced are service marks, trademarks,
or registered trademarks of their respective companies.
Specifications subject to change without notice.
ABOUT TECHNICOLOR
With more than 95 years of experience in entertainment
innovation, Technicolor serves an international base of
entertainment, software and gaming customers. The company is a
leading provider of production, post-production and distribution
services to content creators and distributors. Technicolor is one
of the world’s largest film processors; the largest independent
manufacturer and distributor of DVDs (including Blu-ray discs); and
a leading global supplier of set-top boxes and gateways. The
company also operates an Intellectual Property and Licensing
business unit. For more information: www.technicolor.com
Technical Contact Information


Thierry.Borel@technicolor.com