2020 IEEE 32nd International Conference on Tools with Artificial Intelligence (ICTAI)
A Feasibility Study for Predicting 3D Radiotherapy
Dose Distribution of Lung VMAT Patients
Runxin.Liu Jingfeng.Bai
School of Biomedical School of Biomedical
Engineering Engineering
Shanghai Jiao Tong University Shanghai Jiao Tong University
Shanghai, China Shanghai, China
runxin.liu@sjtu.edu.cn jfbai@sjtu.edu.cn
Cheng.Ni
Radiotherapy Business Unit
Shanghai United Imaging
Healthcare Co Ltd
Shanghai, China
cheng_ni_sjtu@163.com
Abstract—Accurate dose prediction has been proved to be able
to improve radiotherapy planning efficiency. Recently, deep
neural networks have been used in this area and made some
progress. However, existing deep-learning-based methods could
not predict dose distribution accurately for tumors at various
locations, i.e. lung cancer. This article proposes a new deep neural
network CAD-UNet that combines 3D U-net, dense connection,
and SE-net architecture. Spatial distance information is used as a
special input channel in addition to contour information. Dice
similarity coefficients of planning target volume (PTV) region was
added to the mean squared error (MSE) loss function. A cohort of
192 VMAT plans for lung cancer patients was selected for this
study. The trained CAD-UNet and HD-UNet were tested on the
test cases. The dose parameters derived form predicted dose
distribution were used to generate new plans in the treatment
planning system (TPS). The results showed that CAD-UNet can
successfully predict dose distribution of lung cancer cases in
VMAT, outperforming HD-UNet in PTV region homogeneity.
Regenerated plans based on predicted dose showed improvements
in DVHs of organs-at-risk (OAR). Those improvements showed
that CAD-UNet has the potential to guide dosimetrist in the
radiotherapy planning stage.
Keywords—CAD-UNet; deep neural network; radiotherapy;
dose prediction; lung cancer;
I. INTRODUCTION
Lung cancer is the most commonly diagnosed cancer and the
most common cause of cancer death [1]. Radiation therapy is an
important treatment modality for lung cancer and is often used
as adjuvant therapy. About half of lung cancer patients are
advised to receive radiotherapy. It could improve local control,
survival, and patient life quality. The treatment outcome of
radiotherapy for early non-small Cell Lung Cancer (NSCLC) is
comparable to surgical treatment but without operative wound
[2]. Intensity-modulated radiation therapy (IMRT) and
2375-0197/20/$31.00 ©2020 IEEE
DOI 10.1109/ICTAI50040.2020.00197
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 22,2021 at 01:42:54 UTC from IEEE Xplore. Restrictions apply.
Jingjie.Zhou Kang.Zhang
Radiotherapy Business Unit Radiotherapy Business Unit
Shanghai United Imaging Shanghai United Imaging
Healthcare Co Ltd Healthcare Co Ltd
Shanghai, China Shanghai, China
jingjie.zhou@united- kang.zhang@united-imaging.com
imaging.com
volumetric modulated arc therapy (VMAT) are the most
commonly used techniques. Compared to IMRT, VMAT can
achieve better dose distribution, namely higher target dose
coverage and lower organs-at-risk (OAR) dose [3, 4]. However,
VMAT planning is time-consuming and often requires
experienced dosimetrists to tune the dose-volume constraints
manually by trial-and-error.
To overcome this problem, there are several methods
proposed for the automatic treatment planning process,
including Erasmus-iCycle [5], Auto-Planning Engine [6], and
knowledge-based planning (KBP) strategy [7]. And all of these
methods need to predict accurate dose-volume histograms
(DVHs) or voxel-wise dose distribution that serves as
optimization constraints or optimization goals for IMRT or
VMAT planning. And for the reason that voxel-wise dose
prediction provides more spatial information and can be used to
calculate DVH easily, voxel-wise dose prediction seems to have
more potential in automatic planning in radiotherapy. There are
numerous studies on voxel-wise dose prediction [8-11]. These
studies all used neural network and images or 3D volumetric as
input and predicted three-dimensional dose distributions.
However, most studies focus on cases in which tumor sites are
relatively similar among patients. Barragán-Montero [11] tried
to add beam angle information as input to predict lung cancer
dose distribution of IMRT plan and shown relatively high
precision, but for VMAT plans, the beam angle information is
not as important for IMRT plans. This study hoped to be able to
predict VMAT dose distribution for tumor at various locations
without user-specified beam angle.
In this study, we proposed a new model named Channel
Attention Densely-connected U-Net (CAD-UNet)ˈ combining
3D U-Net, SE-net modular, and dense connection into a novel
deep neural network. In addition to contour information of
1304
regions of interest (ROI), spatial distance information is added.
Dice similarity coefficients of PTV was combined with mean
square error (MSE) global loss function. Then the predicted
results were compared with the HD-UNet [10] results. Finally,
we compared regenerated plans using predicted DVHs with the
original manual plans.

II. METHODS
A. Deep neural network
The architecture of the new deep neural network used was Fig. 2. SE-dense-block
proposed by combining 3D U-Net and concatenate operator for
feature incorporation. The new deep neural network architecture
for dose prediction is illustrated in Fig1.
To utilize channels of each layer more efficiently, SE-dense-
block was proposed, as showed in Fig2. The SE-dense-block
incorporates SE-block [16] and dense connection. This block
uses Squeeze and Excitation module for adding channel
attention, followed by a standard 3D convolution, then previous
feature sets are concatenated with new feature sets. In the whole
architecture showed in Fig1, SE-dense-block is alternatively
used with a standard convolutional layer. The number of new
filters was respectively, 32, 64, 128, 256 and 512, with the
feature map size reduced by half after max-pooling layer.
Besides the basic layers of U-net, GroupNorm layer [14] was
added after each convolutional. Rectified Linear Unit (ReLU)
[13] was used as the activation function after the GroupNorm Fig. 3. Spatial distance channel
layer except the last convolutional layer. The proposed model
was implemented in Keras [12] that uses Tensorflow as backend.
Adam optimization algorithm was used to minimize the loss The input network data include 7 channels, and the output
function value between the predicted dose and the original plan data were the 3D predicted dose distribution. The first 6
dose. He_normal [15] initialization method was used to initialize channels were the 3D mask cubes for 6 ROIs, including PTV,
the network parameters. Training batch size, learning rate, and heart, lungs, spinal cord, lungs minus the target, and body.
epochs were set to 2, 10-6, and 100 respectively. Especially, the value of each voxel in PTV mask was set to
prescription dose value while the voxel value of other masks was
set to 1. The last channel represents spatial distance information
to strengthen the PTV location information. This channel
represents the shortest Euclidean distance from each voxel
inside the body to the center point of PTV. The values of voxels
belong to PTV were set to 0. This channel was then normalized,
as shown in Fig3.
To better identify the PTV location, because PTVs of
different cases were scattered all over the lungs and the volume
of PTVs were relatively small compared to the lungs, the Dice
similar coefficient of PTV region, as showed in (1), was
introduced as part of the loss function in the training part.
Fig. 1. The deep neural network architecture. The Black number above blocks
ଶ஽௢௦௘೛ೝ೐೏೔೎೟೐೏ ‫ת‬ெ௔௦௞ು೅ೇ
represent the number of feature maps at the corresponding layer. ܸܲܶௗ௜௖௘ ൌ  (1)
஽௢௦௘೛ೝ೐೏೔೎೐೏ ାெ௔௦௞ು೅ೇ

‘•‡୮୰ୣୢ୧ୡ୲ୣୢ represents the predicted does Cube, and

‫݇ݏܽܯ‬௉்௏ representes the 3D PTV mask Cube.
The global loss function was shown as below (2):

ଵ
௡ ௜ ௜ ଶ
݈‫ ݏݏ݋‬ൌ  ߑ௜ୀଵ ൫‫ܦ‬௣௥௘ௗ െ ‫ܦ‬௢௥௜௚௜௡௔௟ ൯ ൅ ሾͳ െ ሺܸܲܶௗ௜௖௘ ሻሿ (2)
௡

1305

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 22,2021 at 01:42:54 UTC from IEEE Xplore. Restrictions apply.
 The Dpred represented the predicted dose cube and Doriginal
represents the original dose cube. And i is the index of the
voxel and n is the total number of voxels
B. Plan database and Evaluation
Totally, 192 lung cancer cases were used in this study 152
cases for training, 20 cases for validation, and 20 cases for
testing. In this database, the number of patients with tumor in
the left or the right lung were 90 and 102 respectively. All
radiotherapy plans were made using Treatment planning system
(TPS) from Shanghai United Imaging Healthcare with initial
optimize constraints as TABLE Ⅰ followed by several manually
adjustments. All plans satisfied clinical requirements with 95%
PTV region covered by prediction dose, 50Gy. All structure
contours and VMAT plan dose distribution were resized to 256
h256 at each CT slice and each voxel resolution was set to
2.5mmh2.5mm. The CT slice thickness is 3mm. Patch input
was used in this model and patch size was set to 256h256h16
for each input channel, for the large computational cost in the
3D model. Also, patch input could increase training set and play
the role of data augmentation. Hence no other data augmentation
was used for further pre-processing.
஽ଶି஽ଽ଼
‫ ܫܪ‬ൌ 
஽ହ଴
௉ூ௏
‫ ܫܥ‬ൌ 
்௏
To evaluate the accuracy of predicted dose, the dose
distribution, dose-volume histograms (DVHs), homogeneity
index (HI) [17] of PTV, and conformity index (CI) [18] among
CAD-UNet predicted dose, HD-UNet [10] predicted dose and
original plan dose were compared. The definitions of HI and CI
were shown as (3) and (4). And dose-volume constrains
concluded from predicted DVHs, combining initial optimization
constrains, were used to re-plan test cases.
Where Dn means dose received by more than n percentage
of the volume. PIV means prescription isodose volume and TV
means target volume.
III. RESULTS
After 100 epochs, the global loss value between predicted
dose and original dose was decreased from 1.4156 to 0.1486 for
our CAD-UNet, compared the value from 1.5214 to 0.1889 for
HD-UNet. For the 20 test cases, MSE and the average Dice
similarity coefficients were 0.1647f0.0982, 0.5864f0.2584,
93.6543%f4.6547% and 92.5844f6.1549% separately for
two models, taking original plan dose as the baseline.
Fig4 shows the dose distributions from original plans, CAD-
UNet predictions and HD-UNet predictions for two testing cases
with different tumor locations, the unit of the color bar was Gy.
Test case 1 and Test case 2 have different PTV locations in left
lung and right lung separately. As observed from Fig4, two
models can both predict dose distribution for various PTV
location cases.
In addition, the CI and HI of PTV for two model results and
original plan were calculated to evaluate the PTV predicted
results. As observed in Fig5, the test cases results of our model,
showed better dose homogeneity and conformity index in the
PTV region than HD-UNet results. The mean value f standard
deviation of HI and CI for CAD-UNet results, HD-UNet results
and original dose distributions were 0.0993f 0.0369, 0.1969f
(3) 0.0132, 0.0450 f 0.0064, 0.9353 f 0.0620, 0.6194 f 0.0774,
0.9650f0.0124 respectively. The P-values between two model
(4) result for CI and HI were 0.00003182, and 0.00000297
respectively.

TABLE I. INITIAL PLANNING OBJECTIVE SET

Objective
Item(Ii) ROI Target(Ti, Gy) Weight
Type
I1 PTV Max dose ō53 100
I2 PTV Min dose Ŏ50 80
I3 PTV Min DVH D95>50.5 100 Fig. 4. Comparison of dose distribution between original, and predicted dose
from CAD-UNet and HD-UNet on test cases. (Left) original plan;
I4 Heart Max gEUD ō10 6 (Middle) predictions of CAD-UNet; (Right) predictions of HD-UNet;
I5 Heart Max DVH D45>10 5 (Top) Test case 1; (Bottom) Test case 2

I6 Lung-PTV Max DVH D20<20 40

I7 Lung-PTV Max DVH D40<50 10

I8 Lung-PTV Max gEUD ō11 40
Spinal
I9 Max DVH D1<35 8
Cord
Spinal ō38
I10 Max dose 10
Cord
1 the parameter ¢ of gEDU is 1.
Fig. 5. Box plots of HI and CI results of original plan, CAD-UNet and HD-
UNet for test cases.

1306

Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 22,2021 at 01:42:54 UTC from IEEE Xplore. Restrictions apply.
 (a)
(b)
(c)
(d)
Fig. 6. Comparison of DVHs derived from original and predicted dose. (Solid
line) original dose; (Dotted line) predicted dose; (a) prediction from CAD-
UNet for Test case 1; (b) prediction from HD-UNet for Test case 1; (c)
prediction from CAD-UNet for Test case 2; (d) prediction from HD-UNet
for Test case 2.
Fig6 compares the predicted DVHs derived from two
methods with the original DVHs for Test Case 1 and Test Case
2. As can be observed, OAR DVHs of predicted dose with the
two methods were close to the original plan’s DVHs, while the
1307
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 22,2021 at 01:42:54 UTC from IEEE Xplore. Restrictions apply.
Fig. 7. Comparison of DVHs derived from original and re-generated plan dose
(Solid line) original dose; (Dotted line) re-generated plan dose; (Upper)
Test case 1; (Below) Test case 2.
predicted PTV DVH of CAD-UNet is closer to the original than
of HD-UNet. It also demonstrates that the new proposed method
achieves better homogeneity performance in the PTV region.
Different PTV locations did not influence the accuracy of DVHs.
This demonstrated that the new proposed model has better
generalization in predicting cases with various PTV locations.
The DVHs of re-generated plans were shown in Fig7. Re-
generated-plans used dose-volume constraints that are derived
from predicted DVHs by physicists, while the constraints used
in the original plan were tuned by physicists manually in trail-
and-error way. The DVH of PTV between original and re-
generated plans showed in Fig6 is almost the same after each
plan dose of 95% PTV region was scaled to prediction dose. And
some improvements could be observed in the DVHs of OAR.
Especially, the DVH of the esophagus of test case 1 of re-
generated-plan is much lower than the original plan, i.e. D12
decreases from 20Gy to 8Gy. And no other significant point of
re-generated plan is higher than original plans as presented two
cases as the other 18 test cases. It is reasonable to conclude that
the predicted dose of the new proposed model has the potential
to guide physicists or dosimetrist to generate better radiotherapy
plans within less time.
IV. DISCUSSION AND CONCLUSION
In the proposed dose prediction model, Dice similar
coefficient of PTV region was introduced in the loss function
and defined as PTV-Dice loss. It is found that the PTV-Dice loss
helped preserving the uniformity of dose distribution for the
PTV region. Without the PTV-Dice loss, dose fluctuations
would appear in PTV regions and affect the accuracy of the
predicted result. That is because of the PTV located in different
places in lung cancer cases. More attention needs to be focused
on PTV mask cube rather than other input mask Cube to better
impart PTV dose information to the deep neural network. Hence,
we applied the PTV-Dice loss to improve prediction results.
Identically, spatial distance information was implicated as input
to better identify spatial information between PTV to the skin
and other OAR.
Another important innovation of the proposed model is the
use of SE-dense-block to strengthen weights of some channels
to increase training efficiency. It is found that HD-UNet result
showed unsmooth dose distribution in PTV and other region. It
is found that using dense connection alone involves too many
redundancy information in training process and might need
more training epochs to eliminate these redundancy information.
Although HD-UNet performed better than original Dense-Net,
(d)
it still could not predict lung cases well. Because the PTV in lung
cases are at various location. The network needs to concentrate
more on information of PTV region. In other radiotherapy tumor
cases, the tumor locations were similar among different patients
[8-11]. Although the input channel includes spatial information
of PTV region, it was not enough in predicting complicate cases.
Nguyen [10] use HD-UNet to predict cancer in head and neck,
in which tumor locations were almost in the middle of the head
in 2D image. The HI of their results were 0.08f0.02, which is
close to ground truth 0.06f0.04. However, as shown in Fig5,
HD-UNet could not achieve similar good results in the lung case
in this study. The newly proposed CAD-UNet successfully
improved the values of HI and CI to be closer to ground truth
than HD-UNet. Although the standard deviation of our HI and
CI results was a little larger than HD-UNet, our worst result is
still better than best HD-UNet result.
Fig6 results indicated that, by taking predicted DVHs into
consideration, physicists could achieve a lower OAR dose with
fewer times manually tuning and keep the PTV dose index. That
might because some original plans were not pushed to the
physical limits, due to lack of planner experience and
insufficient planning time. Our predicted dose has the potential
to serve as a guideline for planners to achieve better radiotherapy
plan in a shorter time.
The performance of the predicted dose still is not very
precise, especially for those areas away from the PTV region,
which could lead to a not very good result for some auto-
planning methods that use voxel dose rather than DVHs as input.
The bad performance might be caused by many reasons. One of
the main reasons is the quality of the training data. The judgment
of radiotherapy plan quality is subjective and patient-specific,
and differ from physicist to physicist and patient to patient.
Another reason is the limited training cohort size of less than
200, which may impair the ability for the network to predict
complicate cases.
In conclusion, this study proposes a new deep learning
network CAD-UNet that combines densely connected channel
attention with U-Net architecture. In addition to contour
information as input, the spatial distance information was added
as an extra input channel and PTV-dice as part of the global loss
function. The results demonstrates better homogeneity on dose
distribution of PTV regions than HD-UNet and some DVH
improvement from the original plan to re-generated plan
indicates that predicted dose has the potential to guide doctors,
physicists, and dosimetrists in the radiotherapy planning stage.
1308
Authorized licensed use limited to: San Francisco State Univ. Downloaded on June 22,2021 at 01:42:54 UTC from IEEE Xplore. Restrictions apply.
ACKNOWLEDGMENT
Thanks to Dr.Supratik Bose for his insightful advice on
experiment design and manuscript revising.
REFERENCES
[1] Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A. Global
cancer statistics 2018: GLOBOCAN estimates of incidence and mortality
worldwide for 36 cancers in 185 countries. CA Cancer J Clin.
2018;68(6):394-424.
[2]
[3] Mathieu D, Campeau MP, Bahig H, Larrivee S, Vu T, Lambert L, et al.
Long-term quality of life in early-stage non-small cell lung cancer patients
treated with robotic stereotactic ablative radiation therapy. Pract Radiat
Oncol. 2015;5(4):e365-73.
[4] Otto K. Volumetric modulated arc therapy: IMRT in a single gantry arc.
Med Phys 2008; 35: 310-17.
[5] YAO C H, CHANG T H, TSAI M J, et al. Dose verification of volumetric
modulation arc therapy by using a NIPAM gel dosimeter combined with
a parallel-beam optical computed tomography scanner 㹙J㹛. J Radioanal
Nucl Chem, 2017, 311(2): 1277-1286.K. Elissa.
[6] Breedveld, Sebastiaan, Storchi, Pascal R. M., Voet, Peter W. J., and
Heijmen, Ben J. M. Wed . "iCycle: Integrated, multicriterial beam angle,
and profile optimization for generation of coplanar and noncoplanar
IMRT plans". United States. doi:10.1118/1.3676689.
[7] Xhaferllari I, Wong E, Bzdusek K, Lock M, Chen JZ. Automated IMRT
planning with regional optimization using planning scripts. J Appl Clin
Med Phys. 2013;14:176–191.
[8] Dosimetric features-driven machine learning model for DVH prediction
in VMAT treatment planning
[9] Ma M, Kovalchuk N, Buyyounouski MK, Xing L, Yang Y. Incorporating
dosimetric features into the prediction of 3D VMAT dose distributions
using deep convolutional neural network. Phys Med Biol.
2019;64(12):125017.
[10] Song Y, Hu J, Liu Y, Hu H, Huang Y, Bai S, et al. Dose prediction using
a deep neural network for accelerated planning of rectal cancer
radiotherapy. Radiother Oncol. 2020;149:111-6.
[11] Nguyen D, Jia X, Sher D, Lin MH, Iqbal Z, Liu H, Jiang SB Three-
dimensional radiotherapy dose prediction on head and neck cancer
patients with a hierarchically densely connected U-net deep learning
architecture. Phys Med Biol 2019
[12] Barragán-Montero, A. M., Nguyen, D., Lu, W., Lin, M. H., Norouzi-
Kandalan, R., Geets, X., Sterpin, E. & Jiang, S. B., Jan 1 2019, In :
Medical physics. Three-dimensional dose prediction for lung IMRT
patients with deep neural networks: robust learning from heterogeneous
beam configurations
[13] Chollet F 2015 keras: deep learning library for Theano and
tensorflow(https://keras.io)
[14] Glorot X, Bordes A, Bengio Y. Deep Sparse Rectifier Neural
Networks2010.
[15] Y. Wu and K. He. Group Normalization. In The European Conference on
Computer Vision (ECCV), 2018.
[16] K. He, X. Zhang, S. Ren, and J. Sun. Delving deep into rectifiers:
Surpassing human-level performance on imagenet classification. In ICCV,
2015a.
[17] Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation networks. In
Proceedings of the IEEE conference on computer vision and pattern
recognition, pages 7132–7141, 2018.
[18] International Commission on Radiation Units and Measurements. ICRU
83. Prescribing, recording and reporting photon beam intensity-
Modulated Radiation therapy (IMRT). Report No: 83. Washington DC.
2010.
[19] Feuvret L, Noël G, Mazeron JJ, Bey P. Conformity index: a review. Int J
Radiat Oncol Biol Phys 2006; 64: 333–42. doi: https:// doi. org/ 10. 1016/
j. ijrobp. 2005. 09. 028