1

Medical Dosimetrist perceptions about the efficiency and consistency of auto-contouring

during treatment planning: A Survey

Amber Coffey BS, RT(R)(T)(CT); Jose Moreno, BS, RT(T); Muath Ayyad, BS/BA; Nishele
Lenards, PhD, CMD, RT(R)(T) FAAMD; Ashley Hunzeker, MS, CMD; Matt Tobler, CMD,
RT(T)

Medical Dosimetry Program at University of Wisconsin, La Crosse, WI

Key Words: Auto-contouring, Manual contouring, Deep learning, Atlas-based algorithm

Introduction

Advancements in computerized treatment planning systems (TPS) have been substantial,

as evidenced by the multiple tools refined to create a more sophisticated and improved plan.
Computer-aided delineation algorithms are widely available and can save time by reducing the
need for manual-delineation.1 In the last 20 years, auto-contouring capabilities have been
developed using atlas-based methods and deep learning algorithms. It is imperative that
contouring organs at risk (OAR) and target volumes be as accurate as possible to ensure the
radiation dose is targeting the tumor while minimizing dose to healthy tissue and organs.2
However, the quantity of time spent on contouring strongly depends on the experience of the
radiation oncologist and medical dosimetrist.3 As part of the medical dosimetrist’s
responsibilities, contouring is essential to the time saving and efficient workflow necessary for
treatment planning.

Although the efficiency of computer-generated structures is a key benefit in auto-

contouring, several auto-contouring algorithms vary in their individual contouring abilities. The
atlas-based approach uniquely relies on the availability of 1 or more CT series of a specific
anatomical dataset, which has been contoured by a physician following clinical guidelines.4 The
goal of this software is to perform a deformable registration of the atlas subject to transfer to the
CT series that is being contoured. A study shows that contours obtained through atlas-based
algorithms can require inconsequential to substantial editing.4 While some findings indicate that
atlas-based algorithms are yet to replace the human observer in contouring tasks such as soft
tissue in the pelvis region, it does allow for assessing and deploying progressively improved
 2

tools.5 There are diversified results in terms of overall efficiency with atlas contouring methods,
which depends on anatomical regions.5

The deep learning algorithm approach provides an advantage in that the software
automatically generates the most suitable model. Deep learning methods can be trained by
passing a large set of contoured images through convolution neural networks (CNN). This
process allows the algorithm for auto-contouring methods to vary in its consistency of post-
processing contouring technique. Using deep learning contouring significantly reduces the time
in which OAR are contoured.6 Deep learning-based auto segmentation can be clinically useful
when applied to contoured structures with tendencies of increased movement.

Researchers have measured differences between auto-contouring algorithms and manual

contours specific to the delineation of OAR. These studies, with quantitative and qualitative
measures, rationalize the use of auto-contouring algorithms, however, the factor of clinical
workflow was not assessed.7,8,9 While auto-contouring methods can reduce the workload, there
are concerns on whether using auto-contouring will save time if medical dosimetrist’s require
editing with manual contouring.7 If a shortfall in the accuracy of auto contouring arises,
necessary edits must be contrived on every single slice, increasing the potential for time lost.
Manual contouring is known to be time consuming and has considerable variability with
contouring of many structures which may prove daunting.10 While many auto-contoured OAR
and target structures still require manual corrections to be made, researchers showed efficient
time saving ability compared to full manual contouring.1 

Mean durations for initial contours generated took 10.9 minutes, 1.4 minutes, and 1.2
minutes for manual, deep-learning, and atlas-based, respectively. Initial deep learning contours
were more geometrically similar to initial manual contours. Mean durations of contour editing
for manual, deep-learning, and atlas-based contours were 4.1 minutes, 4.7 minutes, and 10.2
minutes, respectively.7 The extent of required editing was larger for atlas-based contours when
compared with manual contouring and deep learning. Using the deep-learning algorithm for
bladder and rectum contours reduced overall time spent on contouring and editing when
compared to atlas-based and manual contouring, without negatively affecting editing times,
contour geometry, or clinically relevant dose-volume metrics.
 3

A study by Andrianarison et al11 showed that it took an average of 7 minutes per structure
to contour, these studies were conducted by a senior radiation oncologist through manual
contouring. Looking at the average time for contouring for prostate patients while utilizing auto-
contouring algorithm, the study by La Macchia et al12 estimated an average of 41 minutes time
saved for deep learning and an average of 31 minutes for atlas-based. When clinically evaluating
contours for lung cancer between manual contouring and auto-contouring, Lustberg et al13 found
that the total median time saved using auto-contouring was 39% less for atlas-based, and half the
time for deep learning algorithms in contouring, comparing all OAR structures to manual
contouring. An additional study by Chen14 demonstrated that deep learning algorithm provides
accurate, consistent and reproducible mastication muscles without the need of any manual
correction. Deep learning algorithms validated auto-contouring for the mastication muscle in
head and neck patients, improving the consistency and efficiency in the radiation therapy
treatment planning workflow compared to manual contouring.14

The problem is there is potential for time and efficiency to be diminished during the
treatment planning process through manual contouring methods. There is a lack of literature
regarding perceptions of auto-contouring benefits. In treatment planning, automation of
contouring can save time by generating organ contours swiftly, which can otherwise be
considered time consuming. The perceived benefits of manual contouring methods are time and
efficiency. Through continued advancements in treatment planning, we have experienced an
increase in the use of auto-contouring tools. The purpose of this study is to investigate medical
dosimetrist’s perceptions of consistency and efficiency treatment planning workflow on auto-
contouring algorithms. The research questions used to guide this study include: (Q1) what are
medical dosimetrist’s perceptions of consistency on auto-contouring algorithms and (Q2) what
are medical dosimetrist's perceptions on treatment planning efficiency when auto-contouring is
used? Measuring both the accuracy of contours and availability of auto-contouring algorithms,
researchers can survey the impact auto-contouring has on the treatment planning process. This
also allows an equitable comparison of auto to manual contouring. By creating a survey, we
measured the medical dosimetrists perceptions of consistency and efficiency of auto contouring
on the treatment planning process.
 4

Methods and Materials 

Survey Selection

An anonymous quantified pilot survey was created in Qualtrics in order to gain

understanding into medical dosimetrist perceptions. Specifically, how consistency and efficiency
of auto-contouring affects the treatment planning process. A 10-question pilot survey was
created with survey questions categorized into 2 types.

Study Population

The study was created through several pretest recapitulations to appropriately format the
study questions. The respondent population consisted of American Association of Medical
Dosimetrists (AAMD) members; the population most likely to have experience with auto-
contouring software. Participation in the survey was limited to active Certified Medical
Dosimetrist (CMD), regardless of experience to collect responses from those most involved in
treatment planning. A survey was distributed through the AAMD network.

Study Design

Both groups of questions were limited to current CMD’s who have previously
implemented auto-contouring. The first group of questions were directed to assess the
perceptions of consistency on the auto-contouring process. The second group of questions
focused on the efficiency of auto-contouring during treatment planning. The questionnaire was
estimated to take no longer than 5 minutes to complete. By creating a survey, researchers were
able to better understand perceptions as to what factors of auto-contouring affected users’
satisfaction with the software, potentially leading to its implementation and further utilization.

Data Collection

Data will be collected once survey responses are returned.

Statistical Analysis

Statistical Analysis will be sent to the statistical consulting center once survey data has
been quantified.
 5

References

1. Aliotta E, Nourzadeh H, Choi W, Leandro Alves VG, Siebers JV. An automated

workflow to improve efficiency in radiation therapy treatment planning by prioritizing
organs at risk. Adv Radiat Oncol. 2020;5(6):1324-1333.
https://doi.org/10.1016/l.adro.2020.06.012
2. Vaassen F, Hazelaar C, Vaniqui A, et al. Evaluation of measures for assessing timesaving
of automatic organ-at-risk segmentation in radiotherapy.  Phys Imaging Radiat Oncol.
2020;13(P1-6):1-6. https://doi.org/10.1016/j.phro.2019.12.001
3. Wu X, Udupa JK, Tong Y, et al. AAR-RT - A system for auto-contouring organs at risk
on CT images for radiation therapy planning: Principles, design, and large-scale
evaluation on head-and-neck and thoracic cancer cases. Med Image Anal. 2019;54:45-62.
https://doi.org/10.1016/j.media.2019.01.008
4. Casati M, Piffer S, Calusi S, et al. Methodological approach to create an atlas using a
commercial auto-contouring software. J Appl Clin Med Phys. 2020;21(12):219-230.
https://doi.org/10.1002/acm2.13093
5. Jackson P, Kron T, Hardcastle N. A future of automated image contouring with machine
learning in radiation therapy. J Med Radiat Sci. 2019;66(4):223-225.
https://doi.org/10.1002/jmrs.365
6. Ahn SH, Yeo AU, Kim KH, et al. Comparative clinical evaluation of atlas and deep-
learning-based auto-segmentation of organ structures in liver cancer. Radiat Oncol.
2019;14(1):213. https://doi.org/10.1186/s13014-019-1392-z
7. Zabel WJ, Conway JL, Gladwish A, et al. Clinical evaluation of deep learning and atlas-
based auto-contouring of bladder and rectum for prostate radiation therapy. Pract Radiat
Oncol. 2021;11(1):e80-e89. https://doi.org/10.1016/j.prro.2020.05.013
8. Kazemifar S, Balagopal A, Nguyen D, et al. Segmentation of the prostate and organs at
risk in male pelvic CT images using deep learning. Biomed Phys Eng Express.
2018;4:055003. https://doi.org/10.1088/2057-1976/aad100
9. Delpon G, Escande A, Ruef T, et al. Comparison of automated atlas-based segmentation
software for postoperative prostate cancer radiotherapy. Front Oncol. 2016;6:178.
https://doi.org/10.3389/fonc.2016.00178
 6

10. McCarroll RE, Beadle BM, Balter PA, et al. Retrospective validation and clinical
implementation of automated contouring of organs at risk in the head and neck: A step
toward automated radiation treatment planning for low- and middle-income countries. J
of Glob Oncol. 2018;(4):1-11. https://doi.org/10.1200/jgo.18.00055
11. Andrianarison VA, Laouiti M, Fargier-Bochaton O, et al. Contouring workload in
adjuvant breast cancer radiotherapy. Cancer Radiother. 2018;22(8):747-753.
https://doi.org/10.1016/j.canrad.2018.01.008
12. La Macchia M, Fellin F, Amichetti M, et al. Systematic evaluation of three different
commercial software solutions for automatic segmentation for adaptive therapy in head-
and-neck, prostate, and pleural cancer. Radiat Oncol. 2012;7:160.
https://doi.org/10.1186/1748-717X-7-160
13. Lustberg T, van Soest J, Gooding M, et al. Clinical evaluation of atlas and deep learning
based automatic contouring for lung cancer. Radiother Oncol. 2018;126(2):312-317.
https://doi.org/10.1016/j.radonc.2017.11.012
14. Chen, W., Li, Y., Dyer, B.A. et al. Deep learning vs. atlas-based models for fast auto-
segmentation of the masticatory muscles on head and neck CT images. Radiat Oncol.
2020;15:176. https://doi.org/10.1186/s13014-020-01617-0