Professional Documents
Culture Documents
Depth Perception Assessment For Stereoscopic 3D Displays Using Layered Random Dot Stereogram
Depth Perception Assessment For Stereoscopic 3D Displays Using Layered Random Dot Stereogram
ABSTRACT Determining the features of human binocular vision according to given disparity information
has gained significant interest with regard to 3-D-relevant fields. However, subjectively distinguishing
the features is internally ambiguous owing to personal factors and external susceptibility due to viewing-
environmental factors. Moreover, high cost and low reproducibility of the subjective assessment method
restricts its usability in numerous practical applications. To alleviate these problems, this paper presents
a novel visual stimulus, namely layered random dot stereogram (LRDS), and hierarchical paradigm for
vote ratings. Based on the guide depth information in LRDS, viewers could recognize considerably obvious
feature points, and our two-step rating strategy enhanced the reliability and efficiency of the viewing tests.
The experiments prove that our approach successfully reflects the effects of not only personal factors but
also display factors on depth perception. Our approach would be beneficial to the 3-D fields of human factor,
viewer-interactive systems, and product performance evaluation.
INDEX TERMS Depth perception, RDS, VQA, stereo acuity, fusion limit, visual discomfort.
would affect the integrity and execution time of the proposed mechanisms critically linked with VD [1], [3]. Secondly,
approach. one of the prepared LRDS stimuli is randomly selected and
With depth and shape variants, we could then further presented to the viewer. Thereafter, two hierarchical vote
evidently investigate HBV FPs on depth perception. For ratings are conducted as follows. In the first vote, the question
instance, if a test depth is considerably small compared is ‘‘which triangle is connected with the centric circle in the
to viewers’ stereo acuity, subsequently, the viewers could mid layer?’’ Subsequently, viewers should push one of the
not recognize any shape information of the mid layer as four direction buttons corresponding to the correct direction.
in Fig. 2(d). Further, if a test depth exceeds viewers’ binocular If viewers fail to recognize the right direction, it is recom-
fusion limit, the viewers might experience diplopia, and the mended to press the question marked button. In the second
blurred layers of the front and mid are attached together vote, viewers should push one of the lettered buttons corre-
concealing the direction information as shown in Fig. 2(f). sponding to the QoE grade for the presented LRDS. Since the
Otherwise, viewers could recognize the exact depth and direc- first vote semantically filters out the marginal grades of the
tion of the mid layer as shown in Fig 2(e), and give correct conventional five-grade scale, we only use three-scale grades
vote ratings. that describe the quality of depth perception concerning
3D effect and VD. Notice that if viewers provide a wrong
answer in the first vote, the second vote is omitted as a time
saving strategy.
FIGURE 4. Example of LRDS stimulus generation. (a) Shape mask samples that have four shape variants and t depth variants, where intensities of each
layer indicate disparities of RDS pattern. (b) LRDS samples converted to multi-view S3D image format.
levels have the same depth since they are directional variants. single binocular vision. Finally, intensities corresponding to
Therefore, more than two stimulus levels having the same the test disparity levels should be allocated to the pixels in the
depth should belong to the same distributions. From this mid layer.
principle, the additional integrity check for final stereo acuity Users can change the shape and the numbers of direction
and fusion limit are performed with Algorithm 1. variant, but the changes may affect the accuracy and exe-
cution time of viewing tests since it determines the number
Algorithm 1 Integrity Check of the Raw Vote Ratings of stimuli. The number of test stimuli is calculated by the
1: procedure STEREO ACUITY (UiL ) F An integrity check number of test disparity levels multiplied by the number of
for stereo acuity shape variants. In Fig. 4, the number of stimulus levels is four
2: SAi ← minUiL F SAnewi is the current times greater than the practical number of the test disparity
stereo acuity value levels since we used four directional variants in the stimuli.
3: SAnew
i ← min(UiL − {SAi }) F SAnew i is a temporal Fig. 5 shows practical LRDS capture images in a glasses-
stereo acuity value free S3D display (side-by-side type), where two capture
new ← U
4: UiL iL methods were used for the comparison: screen capture
5: While (SAnew i -SAi )≥ 4 or {(SAi
new + 3)%4 − (SA + 3)
i (digital capture) and camera capture (optical capture). Since
%4} < 0 do the left and right images of S3D contents are superimposed
6: UiLnew ← U new − {SA } F SAi is regarded as an
iL i in S3D displays, the screen capture images showed slightly
outliner blurred shapes of the Maltese cross in accordance with dis-
7: SAi ← minUiL new F A stereo acuity is newly parity increase. However, images captured by the camera
updated intending a virtual human eye revealed that there are no
8: SAnew
i ← min(U new -{SA })
iL i noticeable differences except for the random dot distribu-
final
9: SAi ← SAi tion. This implies that any shape and depth information is
final concealed in the proposed LRDS without correct binocular
10: return SAi F The final stereo acuity
value SAi
final depth perception, and pure cognitive tests concerning to depth
perception are possible for given disparity levels.
FIGURE 5. Test LRDS sample comparison. Screen captured and camera captured LRDS samples were compared. Left and right images of LRDS samples
are superimposed when it is screen captured, but one of left and right images is shown when it is captured by a camera.
IV. APPLICATIONS AND EXPERIMENTAL RESULTS To this end, all the other factors, related to the 3D content and
The proposed method and tools would be beneficial to various viewing environment [3]–[7], that can affect the HBV FPs
S3D video industries. For instance, in the case of S3D content were restrained. For this test, four non-expert subjects, aged
production, it can aid stereographers to plan a better depth between 25 and 34 years, attended, and we generated 128 lev-
budget for comfort S3D viewing while providing a DB of els of LRDS. Since four consecutive LRDS levels, with dif-
perceptual tolerances for the use of excessive binocular dis- ferent directions, have the same disparity level, 32 levels of
parity. In the case of manufacturing of S3D devices such as disparity were investigated in practice. The same glasses-free
S3D displays, it can be used for performance evaluation of S3D display was used in the same viewing environment for
their products since the range of valid depth expression or the test. The schematic plot for this experiment is depicted
the space of comfort zone could be a quantitative quality in Fig. 7. The accommodation distance was maintained as
measure. In the case of S3D video broadcasting services, 623 mm, and the mid layers of test LRDS stimuli varied
it can give reliable standards to set guidelines for the safe between 619 mm and 313 mm in terms of depth from the sub-
S3D content viewing. Also, it can be cited as a reference tool jects. The first six depth levels were designed to have the step
in academic fields such as 3D human factor research or med- size of 1 mm for stereo acuity, and the others were intended
ical research. To show the feasibility of the proposed method, to have the step size of 10 mm. The angular disparity can
we demonstrate two explorative experiments in this section be calculated from the given accommodation distance, depth
as follows. levels, and the interpupillary distance where we assumed it as
the average length (65 mm).
A. APPLICATION 1: PERSONAL HBV FP INVESTIGATION Fig. 8 illustrates the experimental results. The left graph
The first experiment was designed to investigate personal shows the raw distributions of the stimulus level belonging to
HBV FPs elicited by inherent characteristics of the viewers. each quality grade, rated in the second vote of the hierarchical
FIGURE 8. Experimental results for the first application. (a) Distributions of the raw votes to each quality grade for four different viewers. (b) Human
binocular vision feature points after outlier removal via proposed integrity check.
FIGURE 9. Distributions of raw votes for each quality grade. Comparison of depth expression of three different types of 3D displays.
rating paradigm. Although the correct depth perception is dynamic range for single binocular fusion. This result indi-
verified in the first vote, the wrong quality evaluation could cates that V3 is less sensitive to disparity changes, and needs
be made for numerous reasons. In this graph, we indicated for special caution of viewing excessive depths.
the unreliable votes with a dotted circle, determined by the Additionally, we can analyze each subject’s certainty
proposed integrity check process. The right graph shows (or could be consistency) of the quality evaluation for per-
the final angular disparities corresponding to the five HBV ceived depth by counting the unreliable vote. In this sense,
FPs after the unreliable vote removal, where the dotted lines V2 registered only three unreliable votes, and can be regarded
indicate the two meaningful FPs, the comfort limit and the as the most-favored tester. On the contrary, V3 registered
fusion limit, commonly reported in the literature. twelve unreliable votes, and thus can be considered as an
In a comprehensive sense, all subjects showed similar inappropriate subject for this kind of DPA tests. In this
trends. This result indicates that the proposed approach is manner, the proposed method and tools can be exploited for
considerably free from the internal ambiguity although the screening inappropriate viewers relevant to the judgment.
subjects are not experts in DPA. Also, the experimental results
confirmed the fact that perceptual HBV FPs slightly differ B. APPLICATION 2: PERFORMANCE COMPARISON
from the statistical HBV FPs [1], [3], [7], [9], [10], [12]–[14] OF 3D DISPLAYS
owing to numerous variants of personal factors. For instance, The second experiment was designed to compare the per-
although the limit of Percival’s zone of comfort [1] (1.0◦ ) formance of 3D displays concerning to the ability of depth
belong the dynamic range of ‘‘C’’ grade, the representative expression. To this end, three expert-subjects participated,
disparity slightly oscillates near 0.68◦ across all subjects. and the representative disparity values at the five FPs derived
Similarly, although the limit of the zone of clear single binoc- by three different glasses-free 3D displays were compared,
ular vision is approximately 2.0◦ [12], [13], the binocular where all the other factors that can affect the results were also
fusion limit in our test is about 2.72◦ on average. In the restrained. The only variants in this experiment are the display
individual results, V2 demonstrated the widest dynamic range factors illustrated in Table 1 such as accommodation distance
for single binocular fusion, which indicates that V2 is the and interocular crosstalk. Therefore, it is regarded that the
most sensitive to disparity changes, and the most tolerant of differences at the five FPs for the same subject are elicited
excessive depths. In contrast, V3 demonstrated the narrowest by the properties of the test displays. The experimental
setup was similar to the first experiment, but test depth range This is consistent with the prior research [3], [7], where
and step sizes were distinctively designed according to the the fusion limit was elevated at low spatial frequencies, and
specifications of the test displays, where step size A and B display C had a low spatial frequency induced by its high inte-
were for stereo acuity and the other FPs, respectively. rocular crosstalk. Also, FP2, FP3, and FP4 could be regarded
The experimental results are illustrated in Fig. 9 and as quantitative quality standards for 3D displays. In this
Table 2. Similar to the right graph of Fig. 8, Fig 9 shows perspective, display B demonstrated the highest disparities
the raw distributions of the stimulus level belonging to each at FP3 even for subjects V6 and V7. This result indicates that
quality grade. Since the interests of this experiment are char- display B has the widest zone of comfort.
acteristics of the test displays, unreliable votes were not
highlighted in Fig. 9. Instead, the exact angular disparities for
V. CONCLUSION
each FPs are summarized in Table 2 after the unreliable vote
removal. In conclusion, we presented a novel method and tools for sub-
jective depth perception assessment. We especially focused
TABLE 2. Results of HBV features in the second application. on the fact that viewers could provide wrong votes owing
to the following factors: errors in ratings, guessed votes,
biased judgment, changes in concentration, etc. To this end,
stimuli eliciting inaccurate depth perception are filtered prior
to their quality evaluation, and additional integrity for vote
ratings is checked using the proposed LRDS stimulus. The
cost efficiency is enhanced by our presentation paradigm of
hierarchical vote ratings. The whole processes, a stimulus
presentation to the computation of HBV FPs, are conducted
automatically. Thus, non-expert viewers can assess their own
FPs of depth perception without external aid. To show the
usability for various applications, we also demonstrated two
explorative experiments. Being open-source codes, tools for
LRDS generation and stimulus presentation are available
on the first author’s homepage. These would be beneficial
to various S3D video relevant fields such as 3D human
factor research, personalized 3D content service, 3D hard-
ware performance comparison, developing guidelines for safe
3D viewing with only simple manipulations of particular
The range of single binocular vision, stereo acuity to key-parameters according to the objectives of the target
fusion limit, could be referred as an objective performance applications.
metric for 3D displays. In this perspective, display A
demonstrated the maximum narrow range (perception range:
0.040◦ –1.324◦ /test range: 0.017◦ –1.712◦ ) on average. REFERENCES
We assume that this is deduced from the conventional dis- [1] D. M. Hoffman, A. R. Girshick, K. Akeley, and M. S. Banks, ‘‘Vergence–
advantage of multi-view displays. Although multi-view dis- accommodation conflicts hinder visual performance and cause visual
fatigue,’’ J. Vis., vol. 8, no. 3, p. 33, Mar. 2008.
plays can provide motion parallax even for still images, those [2] L. M. J. Meesters, W. A. IJsselsteijn, and P. J. H. Seuntiens, ‘‘A survey
have a disadvantage of surplus blur caused by adjacent-view of perceptual evaluations and requirements of three-dimensional TV,’’
crosstalk, compared to stereoscopic displays. Especially for IEEE Trans. Circuits Syst. Video Technol., vol. 14, no. 3, pp. 381–391,
Mar. 2004.
display B and C, subjects V6 and V7 correctly perceived the
[3] M. Lambooij, M. Fortuin, I. Heynderickx, and W. IJsselsteijn, ‘‘Visual
min-max test depth thus FP1 and FP5 were not evidently discomfort and visual fatigue of stereoscopic displays: A review,’’ J. Imag.
compared, and certain FP1s and FP2s demonstrated the same Sci. Technol., vol. 53, no. 3, pp. 30201-1–30201-14, 2009.
angular disparities. However, subject V5, the most intolerant [4] M.-K. Kang, H. Cho, H.-M. Park, S. C. Jun, and K.-J. Yoon, ‘‘A well-
ness platform for stereoscopic 3D video systems using EEG-based visual
of excessive depths, recognized that display B apparently discomfort evaluation technology,’’ Appl. Ergon., vol. 62, pp. 158–167,
provides higher fusion limit compared to that of display C. Jul. 2017.
[5] F. L. Kooi and A. Toet, ‘‘Visual comfort of binocular and 3D displays,’’ MIN-KOO KANG (GS’10) was born in Ulsan
Displays, vol. 25, nos. 2–3, pp. 99–108, Aug. 2004. City, South Korea, in 1984. He received the
[6] J. Wang, S. Wang, K. Ma, and Z. Wang, ‘‘Perceptual depth quality in B.S. degree in electronic engineering from Inha
distorted stereoscopic images,’’ IEEE Trans. Image Process., vol. 26, no. 3, University, South Korea, in 2008, the M.S.
pp. 1202–1215, Mar. 2017. and Ph.D. degrees in electrical engineering
[7] Y.-Y. Yeh and L. D. Silverstein, ‘‘Limits of fusion and depth judgment from the Gwangju Institute of Science and
in stereoscopic color displays,’’ Hum. Factors, J. Hum. Factors Ergonom. Technology, South Korea, in 2010 and 2015,
Soc., vol. 32, no. 1, pp. 45–60, 1990. respectively.
[8] D. B. Diner and D. H. Fender, Human Engineering in Stereoscopic Viewing Since 2015, he has been a Post-Doctor with the
Devices. New York, NY, USA: Plenum Press, 1993. Center of Imaging Media Research, Korea Insti-
[9] J. Kim et al., ‘‘Glasses-free randot stereotest,’’ J. Biomed. Opt., vol. 20, tute of Science and Technology, Seoul, South Korea. His current research
no. 6, p. 065004, Jun. 2015.
interests include super-multiview video and high-density multiview video
[10] A. S. Percival, The Prescribing of Spectacles, 3rd ed. Bristol, U.K.: J.
signal processing, 3-D human factor, and viewer-interactive video systems.
Wright & Sons, 1928.
[11] M. Wöpking, ‘‘Viewing comfort with stereoscopic pictures: An exper-
imental study on the subjective effects of disparity magnitude and
depth of focus,’’ J. Soc. Inf. Display, vol. 3, no. 3, pp. 101–103,
Dec. 1995. SUNG-KYU KIM received the B.S., M.S., and
[12] C. Sheard, ‘‘Zones of ocular comfort,’’ Optometry Vis. Sci., vol. 7, no. 1, Ph.D. degrees from the Quantum Optics Group
pp. 9–25, Jan. 1930.
of Physics, Korea University, Seoul, South Korea,
[13] I. P. Howard, Seeing in Depth, Basic Mechanisms, vol. 1. Toronto, ON,
in 1989, 1991, and 2000, respectively. Then, he
Canada: Univ. Toronto Press, 2002.
spent 2 years as an Invited Research Scientist,
[14] K. Ukai and Y. Kato, ‘‘The use of video refraction to measure the dynamic
TAO, Japan. In 2001, he was appointed as a
properties of the near triad in observers of a 3-D display,’’ Ophthalmic
Physiol. Opt., vol. 22, no. 5, pp. 385–388, Sep. 2002. Research Scientist with the Korea Institute of Sci-
[15] T. C. Handy, Event-Related Potentials: A Methods Handbook. London, ence and Technology (KIST).
U.K.: MIT press, 2005. He is currently the Principal Research Scientist
[16] H.-O. Li, J. Seo, K. Kham, and S. Lee, ‘‘Measurement of 3D visual fatigue with KIST. As a Senior Researcher, he has led the
using event-related potential (ERP): 3D oddball paradigm,’’ in Proc. 3DTV Visual Media Research and Development Team, KIST, and succeeded in
Conf., Istanbul, Turkey, May 2008, pp. 213–216. developing original technologies allowing viewers to enjoy 3-D displays
[17] M.-K. Kang and S.-K. Kim, ‘‘Personal binocular vision calibration using without wearing special glasses and compromising resolution. His cur-
layered random dot stereogram,’’ in Proc. APSIPA ASC, Jeju, South Korea, rent research interests also include the next-generation 3-D displays, such
Dec. 2016, pp. 1–4. as super-multiview video displays and multi-focus AR/VR head-mounted
[18] B. Julesz, ‘‘Binocular depth perception of computer-generated patterns,’’ displays.
Bell Labs Tech. J., vol. 39, no. 5, pp. 1125–1162, Sep. 1960.