Use of Technology in Dental Education

Contribution of Haptic Simulation

to Analogic Training Environment in
Restorative Dentistry
Marin Vincent, David Joseph, Christophe Amory, Nathalie Paoli, Pascal Ambrosini,
Éric Mortier, Nguyen Tran
Abstract: The aim of this study was to evaluate the contribution of virtual reality to the conventional analogic training environ-
ment and show the complementarity of conventional techniques and virtual reality in the learning of dental students. All 88 first-
year dental students at a dental school in France in early 2019 were randomly assigned to one of two groups: group 1 (n=45) was
assigned to cavity preparations on a haptic simulator (Virteasy) and group 2 (n=43) was assigned to conventional practical work
on plastic analogue teeth (Kavo). Following three training sessions, the students in group 1 took a final exam on the same plastic
analogue teeth exercise. The results showed improvement in the drilling skill of both groups. The simulator-trained group (group
1) had similar results to the plastic analogue-trained group (group 2) in the final test on a plastic analogue tooth. In this study,
virtual reality allowed an assessment based on objective criteria and reduced the subjectivity of evaluations conducted on plastic
analogue teeth. Considering the saving of supervision and teaching time as well as the material gain offered by virtual reality, the
learning methods of haptic simulators are educational options that should be considered by dental educators.
Marin Vincent, DDS, PhD, is Associate Professor, Department of Restorative Dentistry and Endodontics, Faculty of Odontology
of Nancy, France; David Joseph, DDS, PhD, is Associate Professor, Department of Periodontology, Faculty of Odontology of
Nancy, France; Christophe Amory, DDS, PhD, is Associate Professor, Department of Restorative Dentistry and Endodontics,
Faculty of Odontology of Nancy, France; Nathalie Paoli, DDS, is Assistant Professor, Department of Periodontology, Faculty of
Odontology of Nancy, France; Pascal Ambrosini, DDS, PhD, is Professor, Department of Periodontology, Faculty of Odontology
of Nancy, France; Éric Mortier, DDS, PhD, is Professor, Department of Restorative Dentistry and Endodontics, Faculty of
Odontology of Nancy, France; and Nguyen Tran, PhD, is Associate Professor, Operational Director, School of Surgery of Nancy-
Lorraine, France. Dr. Joseph is co-first author of this article, and Dr. Mortier is co-last author. Direct correspondence to Dr. Marin
Vincent, University of Lorraine, Campus Brabois Santé, Faculté d’Odontologie de Lorraine, 7 avenue de la Forêt de Haye, BP
20199, 54505 Vandœuvre-lès-Nancy, France; 0688605764; marin.vincent@univ-lorraine.fr.
Keywords: dental education, restorative dentistry, preclinical skills, educational technology, simulation, haptics
Submitted for publication 3/1/19; accepted 9/20/19; first published online 12/9/19
doi: 10.21815/JDE.019.187

R
estoring damage to mineralized tooth tissues
resulting from caries is a prerequisite stan-
dard of all dentists. Mastering cavity prepara-
tion is taught in every dental school, through various
training strategies that try to address an appropriate
synergy between the theoretical knowledge necessary
for fully understanding clinical procedures and best
manual practices. However, for many reasons, there
are still gaps in the number and the efficacy of re-
sources, training, and acquisition,1 and there are also
calls for medico-surgical institutions to go beyond
the minimum standard and adopt new approaches
that will meaningfully address modern and objective
certification of proficiency.2-5
Virtual simulation or computer-assisted virtual
assistance may be an efficient educational pathway
to achieving a high level of practice. A review of
the literature by Towers et al. in 2019 found that
numerous questions remain concerning the use of
haptic simulation in dentistry, particularly to deter-
mine the roles and place it should be given in dental
education programs.6 According to Al-Saud et al.,
virtual reality combined with instructor feedback
helps optimize the acquisition and retention of certain
basic conservative dentistry skills.7 The use of haptic
simulators coupled with virtual reality, for which the
realism is constantly improving, enables students to
repeat procedures an unlimited number of times. The
cost benefits of this absence of repetition limit were
described by Suebnukarn et al..8 Furthermore, haptic
simulation may represent an ecological advance since
it limits the waste produced by the use of plastic
teeth in traditional simulation methods. Virtual real-
ity could be eventually used to certify the clinical

Published online ahead of print 9 Dec. 2019  ■  Journal of Dental Education e1

 skills of practitioners as part of continuing profes-
sional development.9,10 Since its inception, simula-
tion in dental education has shown better efficacy in
teaching than traditional techniques through more
effective learning and systematic feedback.9 The
results are particularly encouraging as they concern
fields as varied as implantology,10,11 oral surgery,12
prosthetics,13 pediatrics,14 radiology,15 conservation
dentistry,16 and endodontics.8
In restorative dentistry, cavity preparation
concepts have evolved towards a tissue-sparing
approach, requiring a high level of skill and preci-
sion when preparing cavities. As in other dental
disciplines, restorative dentistry can cause iatrogenic
damage, such as breaching of the pulp chamber, loss
of substance on the proximal surfaces of adjacent
teeth, or excessive deterioration of the dental struc-
ture, ultimately resulting in mutilation due to a lack
of control of the surgical procedure. In addition,
understanding, adhering to, and mastering some
of the geometric preparation concepts used for the
preparation of bonded indirect restorations are some
of the psychomotor skills that need to be acquired
during dental education.17 In 2013, Yamaguchi et al.
reported the utility of haptic simulation for perform-
ing cavity preparation on molars in seven students.18
Wang et al. also found that this type of tool was useful
in restorative dentistry.19 However, they identified
several different research topics to be developed,
including, in particular, increasing the task difficulty.
In this context, we opted to use the proximal
cavity in our study (G.V. Black’s Class II cavity),
conventionally used to evaluate the skill level of
students in cavity preparation due to its relative
complexity in terms of both depth and form. This
type of procedure demands an understanding of the
task to be performed, mastery of the tool via good
hand-eye coordination in order not to damage ad-
jacent teeth, and the need to comply with different
preparation heights. Preparation of a Class II cavity
is a more complete and more complex exercise than
conventional Class I cavity preparation. However,
two studies found that students had low proficiency
on these preparations.15,20
The aim of our study was to evaluate the contri-
bution of virtual reality to the conventional analogic
training environment and show the complementarity
of conventional techniques and virtual reality in the
learning of dental students. Analysis was performed
from comparative data extracted from simulators and
from the traditional method via cavity preparation on
plastic analogue teeth.
e2 Journal of Dental Education  ■  Published online ahead of print 9 Dec. 2019
Materials and Methods
In accordance with the University Hospital of
Nancy procedures, this research was submitted to the
Research and Innovation Department and the Ethics
Committee of the University Hospital of Nancy. No
reservations to the publication of this work were is-
sued. The study was conducted at the School of Sur-
gery of Nancy-Lorraine (Lorraine University, France)
and the Faculty of Dentistry of Nancy (University of
Lorraine, France) in early 2019.
All first-year dental students (n=88) of the
Faculty of Dentistry of Nancy were enrolled and
randomly defined. All the students signed an agree-
ment validated by the school board of the University
of Lorraine. Group 1 (n=45) was assigned to cavity
preparations on a haptic simulator (Virteasy; HRV
Simulation, Changé, France), and group 2 (n=43)
was assigned to conventional practical work on
plastic analogue teeth (Kavo; Kavo Dental, Brea,
CA, USA). Following three training sessions, the
students in group 1 took a final exam on the same
plastic analogue teeth exercise. For both groups,
the maximum time for each cavity preparation was
restricted to ten minutes. There was an interval of
one week between each session.
Simulator and Exercises
The haptic simulator Virteasy consists of a PC-
type computer running Windows 7, a touch-screen
control for interacting with simulator software, a pair
of 3D screen stereoscopic glasses (Estar America
ESG6100; BienestarAmerica, Hillstar, OR, USA)
for viewing the 3D scene, a plastic contra-angled
handpiece connected to force feedback arm device
(Geomagic Touch X Haptic Device, Geomagic Inc.,
Morrisville, NC, USA) to transcribe the tactile sen-
sations of drill in the bone, and a foot pedal to start
the virtual handpiece in the simulator. No student
had experienced virtual simulators before this study.
Before the exercise, each student in both groups
experienced a short demonstration on how to use the
simulator, including use of the virtual handpiece,
ergonomic positions, and feedback force provided by
the machine. The exercise on the haptic simulator was
a Black’s Class II cavity on a first right mandibular
molar (#46) consisting of a principal cavity with a
mesial opening and a depth of 4 mm and in a second-
ary cavity (prophylactic extension) with a depth of 2
mm. The second right mandibular premolar was left
in place throughout the cavity preparation.
 Before the first simulator exercise, all students
in group 1 received a briefing with a PowerPoint
presentation concerning the shape imperatives for
a perfect cavity. The three first sessions (G1 E1, G1
E2, and G1 E3) were performed without assistance
provided by the simulators (but with photographs of
the ideal cavity as a model). All cavity preparation
was achieved by means of an identical dental drill
(Ref 830009, diameter: 0.9 mm). After each exercise
on the haptic simulator, each student underwent a
ten-minute debriefing period with the same teacher,
and areas of improvement were clearly explained
before new sessions. The students had to judge for
themselves the quality of their preparations. Then,
the teacher explained how to improve the prepara-
tion scores.
Plastic Analogue Teeth and
Exercises
The students knew the instrumentation (contra-
angle and burrs as well as the material used for the
teeth). However, the exercise (Black’s Class II cavity)
was not known, and the students did not know the
rules of preparation before this date. They did not
acquire the skills before carrying out the training.
The exercise on plastic analogue teeth was the same
as that on the haptic simulator. All students benefitted
from the same briefing procedure with a PowerPoint
presentation displaying required operations and
expected results.
The three sessions (G2 E1, G2 E2, and G2
E3) were performed on plastic analogue teeth. Dur-
ing these sessions, students had a photograph of
the master preparation as a model. The dental drills
were references 830008 and 830010 (diameter 0.8
and 1 mm). After each exercise, each student had an
individual ten-minute debriefing session, and areas
of improvement were clearly explained before new
sessions. In the last (fourth) session, for the final
exam of group 1 on a plastic analogue tooth, a cavity
preparation following the same described protocol
was performed (noted G1 analogue).
Parameters Studied
Objective parameters recorded from the
simulator. For each test, different objective param-
eters provided by the simulator were recorded and
displayed in an Excel spreadsheet: percentage of
tissue removed corresponding to the required target
(between 0 and 100%), noted “inside” in the Virteasy
Published online ahead of print 9 Dec. 2019  ■  Journal of Dental Education
software; percentage of tissue removed outside the
requested target, noted “outside” in the Virteasy
software; formula calculated as “100 – (inside +
outside)”; and drilling time (time in seconds during
which the drill bit is rotated during the exercise).
Subjective parameters recorded from both
groups. For each test and in both groups, different
subjective parameters were evaluated in a double-
blind manner by odontology teachers. The evalua-
tors were the two supervisors who established the
evaluation grid and followed all the students during
the sessions. Then, all the data were recorded in an
Excel spreadsheet: a) outline shape of the cavity:
-2=very bad (no outline shape), -1=bad (outline
shape slightly pronounced), 0=passable (visible
but irregular outline shape), 1=good (outline shape
globally respected), and 2=very good (outline shape
perfectly respected); b) respect of depth (2 mm)
within the principal and secondary cavities: -1=no
difference (depth=0), 0=depth partially respected
(depth <2 mm), and 1=perfect depth difference
(depth=2 mm); c) regularity of the cavity floor:
-2=over 3 irregularities or presence of a perforation,
-1=3 irregularities <1 mm or presence of 1 irregular-
ity >1 mm, 0=2 irregularities, 1=1 irregularity, and
2=0 irregularity; d) iatrogenic milling on the molar
(#46): -1=iatrogenic milling on 46, and 1=no iatro-
genic milling on 46; and 3) distal iatrogenic milling
on the premolar (#45): -1=iatrogenic cavity on 45,
0=iatrogenic damage on 45 without cavity, and 1=no
iatrogenic damage on 45.
Statistical Analysis
The results were expressed as mean±standard
deviation. Data were analyzed using non-parametric
Friedman statistical tests with Dunn’s correction at
α=0.05 for multiple paired comparison and non-
parametric Kruskal-Wallis statistical tests with
Dunn’s correction at α=0.05 for multiple unpaired
comparison. A probability p<0.05 was considered
significant. Analyses were made possible through
the GraphPad Prism software (GraphPad Software,
San Diego, CA, USA).
Results
The objective outcomes of group 1 are shown in
Figure 1. The analysis of the objective data collected
revealed progressive improvement in students’ drill-
ing skill, especially concerning the crucial formula
e3


Figure 1. Evolution of objective parameters: % inside of group 1 on haptic simulation (panel a); % outside of group 1
on haptic simulation (panel b); formula (100 – inside + outside) of group 1 on haptic simulation (panel c); drilling time
of group 1 on haptic simulation (panel d); total time of group 1 on haptic simulation (panel e); total time of group 2 on
plastic analogue tooth (panel f)

*Significant at p<0.05; **p<0.01; ***p<0.0001

e4 Journal of Dental Education  ■  Published online ahead of print 9 Dec. 2019

of “inside/outside” parameters (p<0.01) (panel c).
The time spent by group 1 to perform virtual drill- Discussion
ing was 216±107 seconds during the first exercise
The current paradigm of surgical training in
decreasing significantly to 150±60 seconds (p<0.01)
general and particularly in dentistry continues to
during phase 3 of training. In parallel, we noted an
raise concerns about the appropriate evaluation of
improvement in the total exercise time (527±102 to
a student’s proficiency since subjective criteria still
425±112; p<0.0001). When considering the novices
play a fundamental role in judgment. Virtual reality in
using the plastic analogue tooth (group 2), we also
the field of medico-surgical training has been shown
observed an improvement in the sole objective pa-
to be an efficient pedagogical adjunct to better shape
rameter available: the total exercise time (543±73 to
the learning process and learning outcomes: simula-
424±105; p<0.0001). Their learning curve is shown
tors with haptic arms have been successfully tested
in panel f.
in many dental and medical fields.21-29 In comparing
Figure 2 shows the gain in experience stem-
this innovative learning method of cavity preparation
ming from subjective outcomes of novices working
with the traditional technique on plastic analogue
with the virtual and analogic cavity models. For the
teeth, several key messages might be extracted
“outline shape of the cavity” and “respect of the
from our study. First, the haptic simulator allowed
depths” parameters, a clear progressive improvement
us, according to available objective parameters, to
over time was observed (p<0.01 for both groups be-
monitor the progression of novices concerning cav-
tween E1 and E3) on virtual training period (group
ity preparation on a Black’s Class II cavity. Second,
1). We also noted a non-significant improvement of
there was no significant difference between the two
the “iatrogenic milling on 46,” “iatrogenic milling on
methods of learning according to the subjective and
45,” and “regularity of the cavity floor” parameters
objective indicators used.
highlighted by a decrease in median and/or SD.
We conducted this study with first-year dental
In comparison, students in group 2, who
students in order to limit potential bias related to
worked with the plastic analogue preparation in the
previous experience. According to Wang et al., it
three training sessions, also had gains in all skills.
would be necessary to increase the difficulty of the
The results were mostly higher with better median
task presented during the research carried out on
and/or SD but non-significant (Figure 3). After the
these simulators.19 For this reason, we used a Black’s
three sessions of both groups, group 1 conducted
Class II cavity to test the progression in dexterity of
the same plastic analogue tooth exercise. When we
first-year students in cavity preparation. This model
tried to compare the two groups on plastic analogue
provides a good base for the development of fine
preparations, the outcomes of all subjective param-
motor skills such as the respect of shape and depth
eters were reported, and no significant difference was
of the cavity, the regularity of the cavity floor, and
found between the two groups. However, only based
the absence of iatrogenic milling. Likewise, without
on median and/or SD, we can note that the results of
well-integrated psychomotor skills such as timing,
group 1 were equivalent to or better than the second
sequencing, speed and precision of hand-eye coordi-
exercise of group 2. However, the analysis of the
nation, and mechanical ability, proficiency in cavity
“iatrogenic milling on 45” parameter on the plastic
preparation becomes quite challenging, and some
analogue tooth showed lower results for group 1 than
authors have called for a better certification of this
for the three exercises of group 2.
procedure.7,30
Regarding the sole comparable objective
A major factor driving an efficient learning
parameter—the total exercise time—we found a
curve is likely to be the reliance on correlation in du-
significant difference between the plastic analogue
ration of procedure and in performance of dexterity,
exercise of group 1 and the second and third exercises
which is a fundamental component of efficiency.10,31-36
of group 2 (p<0.01 and p<0.001, respectively). Even
Our findings clearly showed that the two methods
if the exercise was the same on the plastic analogue
of cavity preparation (virtual or physical) led to
tooth and virtual reality, the transition from a virtual
improvement in dexterity with similar beneficial
to an analog environment for the students of group
impact on the learning curves. Indeed, the time spent
1 would explain this no significant difference in
performing the exercises diminished significantly
total time between the two first analogue exercises
with training in both groups. Regarding group 1,
of both groups.
the time for exercise completion was significantly

Published online ahead of print 9 Dec. 2019  ■  Journal of Dental Education e5



Figure 2. Evolution of subjective parameters and total exercise time with group 1 on haptic simulator (left) and group 2
on plastic analogue tooth (right): outline shape of the cavity (panel a); respect of the depths (panel b); regulatory of the
cavity floor (panel c); iatrogenic milling on 45 (panel d); iatrogenic milling on 46 (panel e); total time (panel f)

*Significant at p<0.05; **p<0.01; ***p<0.0001

e6 Journal of Dental Education  ■  Published online ahead of print 9 Dec. 2019



Figure 3. Comparison between groups 1 and 2 on plastic analogue tooth: outline shape of the cavity (panel a); respect
of the depths (panel b); regularity of the cavity floor (panel c); iatrogenic milling on 45 (panel d); iatrogenic milling on
46, no SD for G1 analogue, all students had the note 1 (panel e); and total time (panel f)

*Significant at p<0.01; **p<0.001

Published online ahead of print 9 Dec. 2019  ■  Journal of Dental Education e7

 reduced, going from an average of 216 seconds in
the first exercise to 150 seconds in the third one—an
improvement of about 30%. To further evaluate their
precision in achieving a virtual cavity, we used the
additional parameter “formula 100 – (% inside + %
outside)” to better define the student’s improvement
than the “% inside” and “% outside” criteria alone.
Indeed, a student with a “% inside” of 95% could be
considered as good while outcome of “% outside”
was over 10%. This formula allowed us to eliminate
this statistical bias by considering the complete per-
formance of each student. Taken together, we found
a concomitant improvement in ability as witnessed
by the result obtained from the ratio between the
percentages of “inside” and “outside” of students
in this group displaying a significant learning curve
during the training phase.
As with group 1, reduction in timing was also
seen in the group of students undergoing learning on
the plastic analogue materials (group 2)—improve-
ment being 22% after three sessions going from
543±73 to 424±105 seconds. This improvement
was also associated with a significant improvement
in “outline shape of the cavity” and “respect of the
principal and secondary cavity depths.” Thus, it could
be argued that the combination of repetition and de-
briefings during the phase of acquisition was crucial
to reach the expected pedagogical endpoint. It should
be noted that the timing to prepare a virtual cavity
was very similar to the time spent for plastic analogue
achievement (Figure 2). However, the simulator-based
students performed cavity preparation with less iatro-
genic damage. We theorize that this outcome might be
attributable to the enhanced computerized visual sys-
tem favoring the hand-eye coordination of the novice.
We and others have previously examined the feasibil-
ity and benefits of 3D simulators on task-oriented
approaches in new surgical techniques such as robotic
surgery37,38 and implantology.10,11 Magnification of
details helps strengthen the cognitive acquisition of
the task and improves the confidence of the novice.
Prior studies found that young residents with former
experience in microsurgery (3D visualization and
coordination) significantly improved their abilities
and their performance in robotic training.39,40 We
suggest the use of a dedicated microscope in cavity
preparation training might be an effective tool to
contribute to enhancing the outcome.
In our study, the virtual reality and conventional
learning curves were extremely similar. However,
the results obtained on simulators were slightly
better. Associated with the saving of teaching and
e8 Journal of Dental Education  ■  Published online ahead of print 9 Dec. 2019
supervision time, the material gains, and the repeat-
ability of the acts, virtual reality could be a real asset
for the education of dental students.
Interestingly, new findings emerged when stu-
dents in group 1 performed their first plastic analogue
tooth exercise after their virtual training period on
haptic simulators. When we compared their final ana-
logue cavity preparation with the three exercises of
group 2, there was no significant difference, suggest-
ing that earlier training on a virtual cavity was effec-
tive at improving their dexterity when experiencing
a physical model for the first time. Our data from the
final plastic analogue test pointed to the usefulness
of the simulator in the acquisition of surgical skills.
Piromchai et al. found that virtual reality-based train-
ing significantly improved the surgical ability and psy-
chometric skills of learners.41 Simulators with haptic
arms have been successfully tested in many dental and
medical fields.21-28 Our results were in agreement with
these studies and showed that these new virtual real-
ity training technologies could be used in preclinical
training for dental students, allowing them to enhance
their learning curve, as found by Buchanan.42
Similar to Fanning and Gaba,43 we outlined the
importance of mentoring through correct briefing/
debriefing procedure. In the contemporary context
where our educational mission may be hampered by
low teacher-student ratios, it becomes more and more
difficult to maintain an appropriate level of interac-
tion between learners and tutors. The objective of
the implementation of this new technology would be
therefore to complete the training of dental preclinical
students.1 Prior studies have found a real benefit in
terms of time, individualized training, and material
and human resources.44-46 In these ways, students can
be trained more quickly with systematic feedback,
and their progress curves are accelerated. In addition,
self-service concepts could allow students to be able
to train outside of their preclinical hours, without the
mandatory presence of teachers. In this framework,
and for all the reasons outlined in this study, we agree
with McGaghie et al. that the learning methods of
haptic simulators are educational options that should
be fully considered.47
This study had several limitations. Since it
took place in only one academic dental institution,
its results may not be generalizable to students in
other programs. Also, the study may have had human
factor bias, and it used only one type of simulator
and took place in only one area of dentistry. Future
studies should repeat the research with other groups
of students in order to check its reproducibility, to
increase sampling to reduce human factor bias, to
validate our results with other types of simulators,
and to expand this educational method to other areas
of dentistry (implantology, prosthodontics, etc.).
These studies are already under way.
Conclusion
Our study highlighted the complementarity of
the traditional and virtual method in the training of
dental students and showed that virtual reality pre-
pared students for the analog environment. In learn-
ing phasing, haptic simulation can serve to improve
these aspects of this particular preparation prior to
transitioning to the analog training. Objective certi-
fication of basic dexterity in odontology thanks to
these new devices should gradually become essential
in initial training and continuing education. In order
to be fully usable, the development and implementa-
tion of new objective evaluation criteria could be a
solution to improve student self-assessment.
Acknowledgments
This work was carried out thanks to the support
of the European Fund for Regional Development
(Fonds Européens de Développement Régional,
FEDER) within the framework of the project Virtual
Hospital of Lorraine.
REFERENCES
1. LeBlanc VR, Urbankova A, Hadavi F, Lichtenthal RM.
A preliminary study in using virtual reality to train dental
students. J Dent Educ 2004;68(3):378-83.
2. Kaufmann CR. Computers in surgical education and the
operating room. Ann Chir Gynaecol 2001;90(2):141-6.
3. Wanzel KR, Ward M, Reznick RK. Teaching the surgi-
cal craft: from selection to certification. Curr Probl Surg
2002;39(6):573-659.
4. Okuda Y, Bryson EO, DeMaria S Jr, et al. The utility of
simulation in medical education: what is the evidence?
Mt Sinai J Med 2009;76(4):330-43.
5. Bilotta FF, Werner SM, Bergese SD, Rosa G. Impact and
implementation of simulation-based training for safety.
Sci World J 2013;13:652956.
6. Towers A, Field J, Stokes C, et al. A scoping review of the
use and application of virtual reality in preclinical dental
education. Br Dent J 2019;226(5):358.
7. Al-Saud LM, Mushtaq F, Allsop MJ, et al. Feedback and
motor skill acquisition using a haptic dental simulator.
Eur J Dent Educ, June 1, 2016.
8. Suebnukarn S, Hataidechadusadee R, Suwannasri N, et
al. Access cavity preparation training using haptic virtual
reality and microcomputed tomography tooth models. Int
Endod J 2011;44(11):983-9.
Published online ahead of print 9 Dec. 2019  ■  Journal of Dental Education
9. Perry S, Bridges SM, Burrow MF. A review of the
use of simulation in dental education. Simul Healthc
2015;10(1):31-7.
10. Joseph D, Jehl JP, Maureira P, et al. Relative contribu-
tion of haptic technology to assessment and training in
implantology. BioMed Res Int 2014;14:e413951.
11. Kinoshita H, Nagahata M, Takano N, et al. Development
of a drilling simulator for dental implant surgery. J Dent
Educ 2016;80(1):83-90.
12. Pohlenz P, Gröbe A, Petersik A, et al. Virtual dental
surgery as a new educational tool in dental school. J
Craniomaxillofac Surg 2010;38(8):560-4.
13. Kikuchi H, Ikeda M, Araki K. Evaluation of a virtual real-
ity simulation system for porcelain fused to metal crown
preparation at Tokyo Medical and Dental University. J
Dent Educ 2013;77(6):782-92.
14. Papadopoulos L, Pentzou AE, Louloudiadis K, Tsiatsos
TK. Design and evaluation of a simulation for pedi-
atric dentistry in virtual worlds. J Med Internet Res
2013;15(10).
15. Nilsson TA, Hedman LR, Ahlqvist JB. A randomized
trial of simulation-based versus conventional training of
dental student skill at interpreting spatial information in
radiographs. Simul Healthc 2007;2(3):164-9.
16. Eve EJ, Koo S, Alshihri AA, et al. Performance of dental
students versus prosthodontics residents on a 3D immer-
sive haptic simulator. J Dent Educ 2014;78(4):630-7.
17. Roy E, Bakr MM, George R. The need for virtual reality
simulators in dental education: a review. Saudi Dent J
2017;29(2):41-7.
18. Yamaguchi S, Yoshida Y, Noborio H, et al. The usefulness
of a haptic virtual reality simulator with repetitive training
to teach caries removal and periodontal pocket probing
skills. Dent Mater J 2013;32(5):847-52.
19. Wang D, Zhao S, Li T, et al. Preliminary evaluation of a
virtual reality dental simulation system on drilling opera-
tion. Biomed Mater Eng 2015;26(Suppl 1):S747-56.
20. Zou H, Jin S, Sun J, Dai Y. Cavity preparation skill evalu-
ation system for assessing junior dental students. Med
Educ 2016;50(5):586-7.
21. Gallagher AG, Lederman AB, McGlade K, et al. Discrimi-
native validity of the minimally invasive surgical trainer
in virtual reality (MIST-VR) using criteria levels based
on expert performance. Surg Endosc 2004;18(4):660-5.
22. Gallagher AG, Richie K, McClure N, McGuigan J. Objec-
tive psychomotor skills assessment of experienced, junior,
and novice laparoscopists with virtual reality. World J Surg
2001;25(11):1478-83.
23. Gallagher K, Stephenson J, Brown R, et al. Exploiting 3D
spatial sampling in inverse modeling of thermochronologi-
cal data. Rev Mineral Geochem 2005;58(1):375-87.
24. Wierinck ER, Puttemans V, Swinnen SP, van Steenberghe
D. Expert performance on a virtual reality simulation
system. J Dent Educ 2007;71(6):759-66.
25. Wierinck E, Puttemans V, Swinnen S, van Steenberghe
D. Effect of augmented visual feedback from a virtual
reality simulation system on manual dexterity training.
Eur J Dent Educ 2005;9(1):10-6.
26. van der Meijden OAJ, Schijven MP. The value of haptic
feedback in conventional and robot-assisted minimal
invasive surgery and virtual reality training: a current
review. Surg Endosc 2009;23(6):1180-90.
e9
 27. Quinn F, Keogh P, McDonald A, Hussey D. A study
comparing the effectiveness of conventional training and
virtual reality simulation in the skills acquisition of junior
dental students. Eur J Dent Educ 2003;7(4):164-9.
28. Moorthy K, Munz Y, Sarker SK, Darzi A. Objec-
tive assessment of technical skills in surgery. BMJ
2003;327(7422):1032-7.
29. de Boer IR, Lagerweij MD, de Vries MW, et al. The effect
of force feedback in a virtual learning environment on the
performance and satisfaction of dental students. Simul
Healthc 2017;12(2):83-90.
30. Mirghani I, Mushtaq F, Allsop MJ, et al. Capturing differ-
ences in dental training using a virtual reality simulator.
Eur J Dent Educ, Nov. 1, 2016.
31. Ioannou I, Kazmierczak E, Stern L, et al. Towards defining
dental drilling competence, part 1: a study of bone drilling
technique. J Dent Educ 2010;74(9):931-40.
32. Mohtashami F, von Dadelszen P, Allaire C. A surgical
virtual reality simulator distinguishes between expert
gynecologic laparoscopic surgeons and perinatologists.
JSLS 2011;15(3):365-72.
33. Mandava SH, Liu J, Maddox MM, et al. Stratification of
expert vs. novice laparoscopists using the basic laparo-
scopic urologic surgery (BLUS) curriculum at a single
institution. J Surg Educ 2015;72(5):964-8.
34. Kenney PA, Wszolek MF, Gould JJ, et al. Face, content,
and construct validity of dV-Trainer, a novel virtual reality
simulator for robotic surgery. Urology 2009;73(6):1288-92.
35. Ioannou I, Stern L, Kazmierczak E, et al. Towards defin-
ing dental drilling competence, part 2: a study of cues and
factors in bone drilling. J Dent Educ 2010;74(9):941-50.
36. Morton J, Cumming A, Cameron H. Performance-based
assessment in undergraduate medical education. Clin
Teacher 2007;4(1):36-41.
e10 Journal of Dental Education  ■  Published online ahead of print 9 Dec. 2019
37. Perez M, Perrenot C, Tran N, et al. Prior experience in
micro-surgery may improve the surgeon’s performance in
robotic surgical training. Int J Med Robot 2013;9(3):351-8.
38. Perrenot C, Perez M, Tran N, et al. The virtual reality
simulator dV-Trainer is a valid assessment tool for robotic
surgical skills. Surg Endosc 2012;26(9):2587-93.
39. Bhayani SB, Andriole GL. Three-dimensional (3D) vi-
sion: does it improve laparoscopic skills? An assessment
of a 3D head-mounted visualization system. Rev Urol
2005;7(4):211-4.
40. Tanagho YS, Andriole GL, Paradis AG, et al. 2D versus
3D visualization: impact on laparoscopic proficiency us-
ing the fundamentals of laparoscopic surgery skill set. J
Laparoendosc Adv Surg Tech A 2012;22(9):865-70.
41. Piromchai P, Avery A, Laopaiboon M, et al. Virtual reality
training for improving the skills needed for performing
surgery of the ear, nose, or throat. Cochrane Database Syst
Rev 2015;9:CD010198.
42. Buchanan JA. Use of simulation technology in dental
education. J Dent Educ 2001;65(11):1225-31.
43. Fanning RM, Gaba DM. The role of debriefing in simu-
lation-based learning. Simul Healthc 2007;2(2):115-25.
44. Miller MD. Office procedures: education, train-
ing, and proficiency of procedural skills. Prim Care
1997;24(2):231-40.
45. Derossis AM, Fried GM, Abrahamowicz M, et al. De-
velopment of a model for training and evaluation of
laparoscopic skills. Am J Surg 1998;175(6):482-7.
46. Kalsi AS, Higham H, McKnight M, Dhariwal DK. Simula-
tion training for dental foundation in oral and maxillofacial
surgery: a new benchmark. Br Dent J 2013;215(11):571-6.
47. McGaghie WC, Issenberg SB, Petrusa ER, Scalese RJ.
A critical review of simulation-based medical education
research: 2003-09. Med Educ 2010;44(1):50-63.