TagedFiur TagedEn TagedFiur TagedEn

TagedH1Prognostication for Patients Receiving

Palliative Radiation TherapyTagedEn
TagedPSusan Sun,y Monica Krishnan,z,# and Sara Alcorny#TagedEn

Estimation of patient prognosis plays a central role in guiding decision making for the pal-
liative management of metastatic disease, and a number of statistical models have been
developed to provide survival estimates for patients in this context. In this review, we dis-
cuss several well-validated survival prediction models for patients receiving palliative
radiotherapy to sites outside of the brain. Key considerations include the type of statistical
model, model performance measures and validation procedures, studies’ source popula-
tions, time points used for prognostication, and details of model output. We then briefly
discuss underutilization of these models, the role of decision support aids, and the need
to incorporate patient preference in shared decision making for patients with metastatic
disease who are candidates for palliative radiotherapy
Semin Radiat Oncol 33:104−113 Ó 2023 The Author(s). Published by Elsevier Inc. This is an
open access article under the CC BY-NC-ND license (http://creativecommons.org/
licenses/by-nc-nd/4.0/)

TagedH1IntroductionTagedEn more aggressive intervention6 or more burdensome treat-

ment such as a long course of fractionated palliative RT at

T he selection of management for metastatic disease sites

depends on patient prognosis, and treatment options
may include or be a combination of radiation therapy (RT),
surgery, systemic therapy, or best supportive care. For pallia-
tive RT, dose and treatment modality often vary according to
the intent of treatment, ranging from single-fraction conven-
tional external beam RT to highly conformal stereotactic RT
regimens. Given that life expectancy generally drives treat-
ment intent, radiation oncologists report strong consider-
ation of prognosis when choosing between RT regimens in
this setting.1 Yet numerous studies have demonstrated that
physicians’ unaided survival estimates are generally
inaccurate,2,3 ranging from 20%-60% across studies of
patients with advanced cancer. Oncologists are often overly
optimistic in their predictions for patients near the end of
life,3,4 with 1 systematic review demonstrating that physi-
cians overestimated survival in 9 of 12 included studies for
this patient population.5 Such overestimation can lead to
y interpretation and familiarity with this approach within
TagedEn Department of Radiation Oncology, University of Minnesota, Minneapolis,
MN
z
TagedEn Department of Radiation Oncology, Dana-Farber Cancer Institute/Brigham
and Women’s Cancer Center, Boston, MA
TagedEnThe authors declare that they have no conflict of interest to declare.
TagedEnAddress reprint requests to: Sara Alcorn, MD, MPH, PhD, Department of
Radiation Oncology, University of Minnesota, Phillips-Wangensteen
Building, 516 Delaware St SE, PWB-1, Minneapolis, MN 55455. E-mail:
alcor049@umn.edu
#
TagedEn Contributed equally.
the end of life.7
TagedPSurvival prediction tools specific to patients with meta-
static cancer who are candidates for palliative RT have
emerged over the past 20 years, which could help guide the
selection of prognosis-appropriate palliative RT regimens.
This article will review select well-validated survival predic-
tion models for patients receiving palliative RT to sites out-
side of the brain. Prognostic models for brain-directed
palliative management is out of the scope of this review and
have been discussed elsewhere.8 When considering applying
statistical models to guide clinical care in the context of palli-
ative RT of metastatic disease, important features to consider
include:
TagedEnP(1) Type of statistical model: While a range of statistical
approaches exist for modeling survival times, the most
commonly applied is the Cox proportional hazards
model. Benefits to this approach include relative ease of
the field of medicine. However, Cox models were
designed for use with small datasets, do not scale well
when there are a large number of covariates relative to
the number of observations (n),9 and require that com-
plex interactions and nonlinear effects between variables
be specified a priori.10 Conversely, newer statistical
approaches using artificial intelligence (AI) may circum-
vent many of the limitations of traditional models. For

104 https://doi.org/10.1016/j.semradonc.2023.01.003
1053-4296/© 2023 The Author(s). Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license
(http://creativecommons.org/licenses/by-nc-nd/4.0/)
TagedEnSeminars in Radiation Oncology
example, machine learning models are ideal for large
datasets, can handle inclusion of nonprognostic and col-
linear covariates, and do not require pre-specification of
interactions and relationships between variables. 11 Yet,
model output for AI may be overly complex and difficult
to interpret and can be particularly susceptible to loss of
validity when applied to external populations.12,13 Given
these inherent differences between models, we will dis-
cuss traditional and AI approaches separately in this
review.TagedEn
(TagedP 2) Model performance measures: Performance measures
include discrimination, calibration, overall performance,
reclassification, and clinical usefulness.14 The most reli-
ably reported metric across studies is model discrimina-
tion, which describes the extent to which subjects who
experience the outcome of interest have a higher risk
prediction than subjects who do not experience the out-
come. The concordance statistic (C-statistic or area under
the receiver-operator characteristic curve) generally indi-
cates a model’s discriminative capacity.15 A value of 0.5
indicates the model predicts survival no better than
chance, and a value of 1 indicates perfect prediction.
Equally important to report is the model’s calibration,
which is a measure that compares the predicted probabil-
ities of an outcome to the actual observed rates among
subjects.16 This is generally measured using a “goodness-
of-fit” test. It is noted that many of the studies reviewed
maintain model discriminative capacity but may show
evidence of poor calibration when applied to external
data sets. Models may be poorly calibrated in general or
for specific subsets of the population. Whether poor cali-
bration is meaningful depends on the clinical context.
For example, a model may be poorly calibrated at the
extremes of a prediction range (ie, it under-or overesti-
mates survival for those with very low or very high pre-
dicted probabilities of survival at a given time point) but
clinical decisions may tend to be made based on proba-
bilities near the middle of the range, such as the 50%
probability of survival (ie, median survival for a given
time point). In this circumstance, inadequate calibration
may not be clinically significant, and the model may
maintain its utility.TagedEn
TagedPNewer methods for assessing performance including
reclassification (generally reported in tabular form), overall
model performance (reported as the Brier score), and clinical
usefulness (reported as net benefit or decision curve analysis)
have yet to gain widespread use in this clinical context. For
this review, we will focus on model discrimination and cali-
bration, given that these measures are consistently reported
across most relevant studies.
TagedEnP(3) Validation procedures: Commonly employed methods
such as cross-validation and temporal validation utilize
patients within the same or similar source populations to
estimate model generalizability. Within this review, we
have attempted to highlight models that have specifically
undergone external validation to ensure optimal
105
generalizability to external populations. We did choose
to include select studies of stereotactic RT and AI model-
ing with internal validation only, given paucity of exter-
nally-validated studies for these topics.TagedEn
TagedP(4) Study population: Maintenance of model validity to an
external population may erode if the survival time and
features of the external population vary substantially
from the source population. This may be particularly
important when considering prognostication in patients
who are receiving stereotactic RT. Selection of this
modality is often reserved for patients with limited extent
of disease or anticipated prolonged survival time, calling
into question whether models derived from patients who
received conventional palliative RT remain valid when
applied to stereotactic cohorts. Indeed, evidence suggests
that both traditional statistical models and machine
learning models built from source populations treated
with conventional palliative RT to bone sites may main-
tain discriminatory capacity but may be poorly-calibrated
for use in prognostication for patients receiving stereo-
tactic RT.17 As such, we will consider models for use in
stereotactic management separately below.TagedEn
TagedP(5) Prognostic factors: A variety of different patient prognos-
tic factors are incorporated into the models. Table 1 sum-
marizes some of the commonly used factors.TagedEn
TagedP(6) Time points used for prognostication: Clinically mean-
ingful time points from time of consultation or delivery
of palliative RT until predicted death are 3 months
(sometimes used as a threshold for surgical management
of spinal cord compression18), 6 months (hospice eligi-
bility),19 and 12 months (consideration of dose escala-
tion for improved local control given increased chance of
recurrence and retreatment).20,21 When selecting
between models, users should consider if the time points
reported are applicable to their patient population and
the given clinical scenario.TagedEn
TagedP(7) Model output: Depending on the type of statistical
model and predictive approach utilized, model output may
include: (1) prediction at a specific time point or time
points, (2) classification into categories of survival, or (3)
visualization of a predicted survival curve over time. While
simpler output can enhance ease of clinical use, more com-
plex output permits for prediction at various clinically
meaningful time points. However, such complex output
TagedEnTable 1 Prognostic Factors
Karnofsky This has been demonstrated to correlate
Performance with survival in metastatic cancer.
Status
Symptoms Clinical symptoms frequently used
include weight loss, dyspnea, nausea,
and fatigue.
Laboratory Values frequently used include lactate
values dehydrogenase (elevated in certain can-
cers), C-reactive protein (elevated in
inflammation and cancer), and complete
blood count.
TagedEn106
may require use of special web platforms or
applications.12,22 Users should determine if a given model’s
output is optimal for the specific clinical scenario. The for-
mat of model output also has implications for how these
data are shared with patients, as individual patients may
variably prefer that survival information be presented as
numbers, in words, or in figures such as charts or graphs.23TagedEn
TagedH1Traditional Statistical Models for Use
With Conventional RadiotherapyTagedEn
TagedPTable 2 reviews traditional statistical models that were built
using patient populations managed with conventional pallia-
tive RT. As previously noted, the vast majority of these mod-
els utilize a Cox proportional hazards approach. Key studies
that have been externally validated include Chow’s Number
of Risk Factors, Krishnan’s TEACHH model, Westhoff’s Easy
Tool, Katagiri’s model, and Zaorsky’s METSSS score.22,24-27TagedEn
TagedPChow et al. utilized a Cox regression model containing up
to 6 covariates to predict survival of metastatic cancer
patients who received conventional palliative RT: nonbreast
primary cancer site, presence of metastases other than bone,
Karnofsky performance status (KPS) ≤60, and Edmonton
symptom assessment scale (ESAS) scores for fatigue >4,
appetite >8, and shortness of breath >1.24 The authors then
considered a reduced model comprised of 3 covariates—
nonbreast primary cancer site, presence of metastases other
than bone, and KPS ≤ 60—noting similar model perfor-
mance between the 3- and 6- covariate models. As such, the
authors favored the 3-covariate model for its ease of use, and
survival predictions were made according to the number of
risk factors (NRF) present.28 Those with 0-1 NRF had sur-
vival probability 80, 68, and 53% at 3, 6, and 12 months,
respectively. The 2 NRFs group had survival probability 73,
51, and 26% at 3, 6, and 12 months, respectively. The 3
NRFs group had the worst prognosis with survival probabil-
ity 35, 14, and 3% at 3, 6, and 12 months, respectively. The
C-statistic for this 3-variable NRF prognostic model was
0.65. Calibration curves were not published; however, the
multiple correlation coefficient (R2) for comparison of actual
survival with predicted survival was reported as 0.23. This
value suggests that a fair amount of variance in the survival
outcome was unexplained by the collective predictor varia-
bles. On external validation with 467 patients referred to
Princess Margaret Hospital for palliative RT and evaluated by
the same authors, discriminative ability of the model was
maintained, with a C-statistic of 0.63.28 The generalized R2
was lower at 0.15.28 Chow et al. also used the same dataset
to create a recursive partitioning analysis comprised of KPS
and site of metastases, with model output showing 3 prog-
nostic groups on basis of these covariates.29TagedEn
TagedPThe TEACHH model, which stands for type of cancer,
eastern cooperative oncology group (ECOG) performance
status, age, prior palliative chemotherapy, prior hospitaliza-
tions, and hepatic metastases, describes factors associated
with a shorter life expectancy in patients who received
S. Sun et al.
palliative RT based on Cox regression analysis.27 Specifically,
the risk factors were nonbreast/prostate cancer, ECOG 2-4,
age 60 or older, more than 2 courses of prior palliative che-
motherapy, hospitalization within 3 months of palliative RT,
and presence of liver metastases. Predictions were made
according to number of risk factors; patients with 0-1, 2-4,
and 5-6 risk factors had median survival 19.9 months, 5
months, and 1.7 months, respectively. The C-statistic was
0.59, and calibration was not reported. This model has been
externally validated using retrospective data of 195 patients
referred to a Canadian cancer center for spinal metastasis
RT.30 Discriminative capacity was maintained and improved
as compared to the training set, with a C-statistic of 0.78.
While calibration curves were not included, the generalized
R2 was 0.17, again suggesting a measurable amount of vari-
ance in the survival outcome that was unexplained by the
collective predictor variables in the validation set.TagedEn
TagedPWesthoff et al. used data from the Dutch Bone Metastasis
Study to develop a simple survival prediction tool using Cox
regression analysis.26 Similar to Chow et al., the authors first
developed a more complex model comprised of 6 covariates:
sex, primary tumor type (breast, prostate, lung, or other),
presence of visceral metastases, baseline KPS, visual analogue
scale of general health (VAS-gh), and verbal rating scale of
overall valuation of life (VRS-vl). The authors then compared
the 6-item model performance to a reduced 2-covariate
model utilizing primary tumor type and KPS. As with
Chow’s model, Westhoff et al. found similar performance
with the reduced model and favored its use due to its sim-
plicity. Survival rates at 3, 6, and 12 months were reported
in table format and ranged based on primary tumor type and
KPS value. The C-statistic was 0.71. Calibration curves for
the model showed significant underestimation of actual sur-
vival time, with the best calibration noted for patients com-
prising the worst prognostic groups (eg, those with lung
cancer and poor clinical condition). Given concerns regard-
ing calibration, authors advised that the model be applied
with caution. External validation was performed by the pri-
mary authors on retrospective data from 934 patients who
underwent palliative spine RT at a single institution in the
Netherlands. Discriminative capacity was maintained, with a
C-statistic of 0.72. The validation calibration curves were
similar to those of the source dataset, with overall tendency
toward substantial underestimation of survival.TagedEn
TagedPKatagiri et al. defined a survival model based on patients
with symptomatic bone metastases using Cox regression
analysis.27 Most included patients received conventional pal-
liative RT, but a small number were treated by either surgery
or supportive care alone. Six factors were assigned values,
which were summed to calculate a prognostic score. The fac-
tors included primary site (0 points for slow growth tumors,
2 for moderate growth, 3 for rapid growth), visceral metasta-
sis (1 point for nodular visceral/cerebral metastasis, 2 for dis-
seminated metastasis) and lab data (1 point for abnormality
in C-reactive protein (CRP)/lactate dehydrogenase (LDH) /or
serum albumin, 2 for abnormality in platelet count/serum
calcium/or total bilirubin). Additionally, 1 point was given to
ECOG performance score of 3 or 4, previous chemotherapy,
 TagedEnSeminars in Radiation Oncology
TagedEnTable 2 Traditional Models for Use With Conventional Radiation Therapy
Model Patient Population Model Type Covariates Results Model Performance External Validation
24,28
Chow 395 patients seen for pallia- Cox proportional Primary cancer type, site of metasta- 0-1 NRF: survival C-statistic: 0.65 C-statistic: 0.63
tive RT consultation at a hazards sis, KPS probability 80, 68, and 53% at 3, 6, R2: 0.23 R2: 0.1528
single institution and
Prospectively collected 12 mo
data 2 NRFs: survival
probability 73, 51, and 26% at 3, 6,
and 12 mo
3 NRFs: survival probability 35, 14,
and 3% at 3, 6, and 12 mo
Krishnan; 862 patients who received Cox proportional Nonbreast or prostate primary cancer Patients with 0-1, 2-4, C-statistic: 0.59 C-statistic: 0.78
TEACHH25 palliative RT at a single hazards type, age ≥60, presence of liver and 5-6 risk factors had median sur- R2: 0.1730
institution metastasis, ECOG 2-4, hospitaliza- vival 19.9, 5, and 1.7 mo
Retrospectively collected tion in past 3 mo, >2 courses of
data prior palliative chemotherapy
Westhoff26 1157 patients who received Cox proportional Primary cancer type, KPS Breast primary: KPS 90+, 70-80, 20-60 C-statistic: 0.71 C-statistic: 0.7226
palliative RT for symp- hazards with median survival 20.8, 16.8, 8.1 Calibration: significant Calibration: significant
tomatic bone metastasis mo underestimation of actual underestimation of actual
at multiple institutions Prostate primary: KPS 90+, 70-80, 20- survival time survival time
Prospectively collected 60 with median survival 13.9, 9.1,
data 5.6 mo
Lung primary: KPS 90+, 70-80, 20-60
with median survival 4.7, 3.6, 2 mo
Other primary: KPS 90+, 70-80, 20-60
with median survival 7.2, 4.5, 2.4 mo
Katagiri27 808 patients who received Cox proportional Primary cancer type (score 0 for slow 6 and 12 mo survival rates were 98 Not reported Calibration:
palliative RT for symp- hazards growth tumors, 2 for moderate and 91% for scores 0-3, 74 and 49% 6 and 12 mo survival rates
tomatic bone metastasis growth, 3 for rapid for scores 4-6, and 27 and 6% for were 78 and 61%
at a single institution growth) scores 7-10 for scores 0-3, 49 and 31%
Prospectively collected Visceral metastasis (1 point for nodu- for scores 4-6, and 22 and
data lar visceral/cerebral metastasis, 2 9% for scores 7-1031
for disseminated
metastasis)
Lab data (1 point for abnormality in
CRP/LDH /or serum albumin, 2 for
abnormality in platelet count/serum
calcium/or total bilirubin)
1 point given to
ECOG performance score of 3 or 4,
previous chemotherapy, or pres-
ence of multiple skeletal
metastases
Zaorsky; 68,505 patients who Nomogram; Cox Metastasis location, elderly age, Median survival for low, medium, and C-statistic: 0.71 Correctly categorized
METSSS22 received a first course of proportional primary cancer type, sex, comorbid- high- Calibration: well-calibrated 98.1% of patients who
palliative RT hazards ity, site of RT risk groups were 11.7, 5.1, and 3.3 mo at 1 y, overestimation of died within 2 mo of pallia-
NCDB query actual survival at 5 y tive RT into the high-risk
group32
CRP, C-reactive protein; ECOG PS, eastern cooperative oncology group performance status; KPS, Karnofsky performance status; LDH, lactate dehydrogenase; NCBD, National Cancer Database;
NRF, number of risk factors method; R2, multiple correlation coefficient for comparison of actual survival with predicted survival; RT, radiation therapy.

107
TagedEn108
or presence of multiple skeletal metastases. A total score of 0-
3 had survival rates of 98% and 91% at 6 months and 1 year,
respectively. Scores 4-6 had survival rates 74% and 49% at 6
months and 1 year, respectively. Scores 7-10 had survival
rates 27% and 6% at 6 months and 1 year, respectively. C-
statistic and calibration curves were not reported. On exter-
nal validation, the 6 months and 1 year survival rates were
78% and 61% for scores 0-3, 49% and 31% for scores 4-6,
and 22% and 9% for scores 7-10.31 Model performance
measures were not reported. However, when reviewing the
prediction categories, it appears that the Katagiri system
scores are correlated with survival time but may overestimate
actual survival times in the validation set, particularly for
lower scores.TagedEn
TagedPMore recently, the METSSS score, which stands for
metastases location, elderly age, tumor primary, sex, sick-
ness/comorbidity, and site of radiation therapy, was devel-
oped based on patients from the National Cancer Database
who received a first course of conventional palliative RT.22
The Charlson-Deyo comorbidity score classified the sick-
ness/comorbidity covariate. Cox regression analysis was used
to identify and quantify the 6 risk factors and create a nomo-
gram, which was most influenced by age and type of primary
tumor. Patients were then separated into 3 groups by mortal-
ity risk. The median survival times for low, medium, and
high-risk groups were 11.7, 5.1, and 3.3 months, respec-
tively. The prediction nomogram is also available on a free-
access web platform, with output listing 1- and 5-year sur-
vival predictions based on covariate input. The C-statistic
was 0.71 on temporal validation. The model was well-cali-
brated at 1 year, but error at 5 years was larger due to overes-
timation. An external validation study with patients who
received RT in the last 2 months of life demonstrated that
this model correctly categorized 98.1% (n = 269) of patients
into the high-risk group.32 Details of model performance for
low- and medium-risk groups have not been reported.TagedEn
TagedPTraditional models that are specific for RT target sites
have also been described, notably for spine metastasis with
spinal cord compression33-36 and without neurological
impairment.37 Recent attention has focused on models that
predict short-term mortality to guide management for
patients facing imminent end of life.38-40 Additionally, there
is a model which predicts pain response after palliative RT
for bone metastasis,41,42 and those who respond have an
improved quality of life after RT.43 Of note, many of
these studies have not yet undergone published external
validation.TagedEn
TagedH1Traditional Statistical Models for Use
With Stereotactic RadiotherapyTagedEn
TagedPDose-escalated stereotactic RT offers more durable local con-
trol in comparison to conventionally fractionated RT and
may be advantageous for patients with longer estimated life
expectancy.44 However, stereotactic RT is more expensive,
more resource intensive, and requires more planning time. It
is thus important to accurately identify specific patients who
S. Sun et al.
would most benefit from stereotactic RT. Given the relatively
recent use of stereotactic RT for metastatic disease, especially
for oligometastatic disease, older survival models generally
do not include patients who received stereotactic RT and
may not fully capture characteristics that impact survival or
account for the longer survival times that can be measured in
this patient population. As previously noted, evidence sug-
gests that models built using conventional RT source popula-
tions may maintain discriminatory capacity but tend to be
poorly-calibrated when applied to stereotactic RT popula-
tions.17 Thus, models specific to stereotactic RT patients
may be necessary to predict survival more accurately, espe-
cially as utilization of stereotactic RT for metastases contin-
ues to increase.45 Two models, one specific to stereotactic
RT for spinal metastasis and another specific to stereotactic
RT for lung metastasis, are outlined below and summarized
in Table 3.TagedEn
TagedPThe prognostic index for spine metastases (PRISM) model
described survival predictions for patients that received ste-
reotactic RT for spinal metastases.46 This model was built
through Cox regression. A total of 7 factors were selected
and assigned a score ranging from -2 to +3: score; female
sex: +2, KPS: +1 per units of 10 over 60, previous surgery at
RT site: +2, previous radiation at RT site: -2, RT site as the
only site of disease: +3, number of organ systems involved
other than bone: -1 per system, and >5 years from initial
diagnosis to spine metastasis: +3. A survival score was then
calculated and used to categorize survival prediction into
groups 1-4. Group 1 (scores >7) had the best prognosis
(median survival not reached), while Group 4 (score <1)
had the worse with median survival 9.1 months. Median sur-
vival was 32.4 months for Group 2 (score 4-7) and 22.2
months for Group 3 (score 1-3). The C-statistic for this
model was 0.69 after stratification into subgroups. It is noted
that this model has not been externally validated but is
included in this review in the absence of other externally val-
idated models for use in stereotactic RT populations. On
internal validation with an independent cohort of 249
patients from the same institution, discriminative capacity
was maintained, with a C-statistic of 0.68 after stratification
into subgroups. Calibration showed an overall overestima-
tion of predicted survival as compared to observed survival
in the validation set.47TagedEn
TagedPTanadini-Lang et al. created a survival prediction model
for patients that received stereotactic RT for lung metasta-
ses.48 A nomogram was built based on factors identified
through Cox regression to predict 2-year overall survival.
The factors included KPS, type of primary tumor, control of
disease, size of largest treated metastasis, and number of
metastases. Nomogram score was used to assign patients to
4 risk groups with predicted 2-year overall survival ranging
from 24% to 76%. The C-statistic was 0.73. Regarding cali-
bration, predicted 2-year overall survival versus actual
Kaplan-Meier survival data for the different risk groups were
76.1 vs 78.4%, 62.6 vs 60.5%, 45.8 vs 46%, and 24.2 vs
30.2%, respectively. This model was externally validated
using 2 cohorts of 92 and 145 patients who were treated
with stereotactic RT for oligo-metastatic disease to the lung
TagedEnSeminars in Radiation Oncology 109

at separate institutions. Model discrimination was slightly

tion of actual survival at »50%

predicted survival, underesti-
Calibration: at 2 y, overestima-
Calibration: overestimation of
lower for the respective validation sets, with C-statistics of
Internal validation C-statistic:
0.68 after stratification into

mation of actual survival at

»80% predicted survival48
0.64 and 0.69. Regarding calibration, the model appeared to
overestimate 2-year actual survival when predicted survival

Two validation groups,

actual survival time47
is near 50%, demonstrate good calibration near predicted
Model Performance External Validation

survival of 70%, and underestimate actual survival when pre-

dicted survival is near 80% for both validation sets.48TagedEn
Not available.

0.64 and 0.69

subgroups

C-statistic:
TagedH1Artificial Intelligence ModelsTagedEn
TagedPAs previously discussed, the relative ease of use of traditional
C-statistic 0.69 after
stratification into

statistical models like Cox regressions must be weighed

C-statistic: 0.73

against cons inherent to these types of models, including lim-

subgroups

itations to the number of covariates included relative to the

source population size, the need for a priori specification of
complex relationships between covariates, and lack of
robustness to inclusion of collinear and nonprognostic varia-
(score 4-7), 3 (score 1-3), and
median survival for groups 2

RT site as the only site of disease: +3 4 (score <1) were 32.4, 22.2,

bles. Thus, AI models that are more adept at handling these

(score >7) was not reached;

2-y overall survival ranging

4 risk groups with predicted
Median survival for group 1

methodologic limitations have emerged in recent years in

effort to better predict a patient’s estimated survival relative
to time from RT delivery or consultation. Two such models
are described below and in Table 4.TagedEn
from 76 to 24%

TagedPThe bone metastases ensemble trees for survival (BMETS)

and 9.1 mo

model utilized 27 factors considered to be prognostic for sur-

vival and established an algorithm using random survival for-
Results

ests methodology for predicting survival in patients receiving

conventional palliative RT for bone metastases.12 To enhance
clinical utility of this otherwise complex model, the authors
Number of organ systems involved

>5 y from initial diagnosis to spine

created a web-based platform to facilitate ease of data entry of

tumor, control of disease, size of
largest treated metastasis, and
other than bone: -1 per system
Previous radiation at RT site: -2
KPS: +1 per units of 10 over 60
Previous surgery at RT site: +2

covariate values and interpretability of data output for external

users. The web platform displays a patient-specific predicted
number of metastases

survival curve from time of consultation for palliative radiation

therapy until death from 0 to 12 months.45 Discrimination
KPS, type of primary
TagedEnTable 3 Traditional Models for Use With Stereotactic Radiation Therapy

was measured using the area under the curve (AUC) at 3, 6,

metastasis: +3
Female sex: +2

and 12 months, corresponding to values of 0.83, 0.81, and

Covariates

0.81, respectively. The model remained well-calibrated with

the exception of the extremes of the prediction curves: at 3
months postconsultation, BMETS tends to overestimate sur-
vival for patients with low predicted survival probabilities,
tional hazards

tional hazards

and at 12 months, BMETS tends to underestimate survival for

KPS, Karnofsky performance status, RT, radiation therapy.
Cox propor-

patients with high predicted survival probabilities. Authors

Model Type
206 patients who received Cox propor-

Tanadini- 670 patients who received Nomogram;

accounted for this by noting a prediction range for appropriate

application of the model on their web platform. On external
validation by the same authors with data evaluating 216
Prospectively collected data

symptomatic bone sites in 182 patients from 2 community

PRISM46 stereotactic RT for spine

Retrospectively collected
Lang48 stereotactic RT for lung

radiation oncology clinics not included in the training set,

model discrimination was maintained, with AUC at 3, 6, and
Patient Population

12 months of 0.82, 0.77, and 0.77, respectively. Calibration

curves were also similar to that observed in the source popula-
tion.49 A separate external validation was performed using 326
metastasis

metastasis

patients treated with palliative RT for bone metastases from a

data

single institution in Norway. While the authors of this inde-

pendent validation did not report on model discrimination,
they specifically comment on calibration, noting that BMETS
Model

predicted survival accurately for most patients but that its

Tang;

accuracy did decrease to 68% for patients with actual survival

<3 months.50TagedEn
TagedEn110 S. Sun et al.

TagedPAnother machine learning model from Stanford utilized

Calibration: at 3 mo, overestimation Calibration: at 3 mo, overestimation

of actual survival for patients with
low predicted survival probability
AUC at 3, 6, and 12 mo: 0.83, 0.81, AUC at 3, 6, and 12 mo: 0.82, 0.77,
4126 predictor variables from the electronic medical record

actual survival for patients with

and at 12 mo, underestimation
data to predict survival in patients who received palliative
RT.51 The variables ranged from text from clinical notes to

high predicted survival

ICD-9 diagnosis codes and medications. The bag-of-words
approach of natural language processing and information
External Validation

retrieval was used to extract features from text in the medical

record. The C-statistic was 0.745 for palliative RT courses.
probability.49
Not available
Calibration was generally acceptable, without evidence of
and 0.77

systematic over- or underestimation. This model was not

specific to patients receiving conventional palliative RT and
has not undergone published external validation to date.
Nonetheless, we have included this tool in our review given
high predicted survival probability
of actual survival for patients with

its innovative approach to prognostication and the breadth

low predicted survival probability

evidence of systematic under- or

4126 variables C-statistic: 0.745 for palliative RT
actual survival for patients with
and at 12 mo, underestimation

Calibration: acceptable, without

of covariates evaluated. Given complexity of the model input

and output, its applicability to external users has yet to be
defined.TagedEn
Model Performance

TagedH1DiscussionTagedEn
over-estimation

TagedPPatients with terminal cancer who better understand their

and 0.81

courses

prognosis are more likely to select less aggressive treatment

options,52 and appropriate palliative management is associ-
ated with improved patient quality of life,53,54 lower cost,55
and better surviving caregiver quality of life.53 Patients with
advanced cancer who discuss their life expectancy with their
27 variables
Covariates

oncologists have a better understanding of their disease56

and are more likely to engage in advanced care planning
including drafting wills and specifying power of attorney.57
However, oncologists often fail to discuss estimated survival
with those with poor prognosis.58 There are many reasons
features extracted from text
unstructured data from the

via bag-of-words approach

electronic medical record;

for why physicians may avoid end-of-life discussions, includ-

including structured and
Random survival forests

ing their lack of accuracy and confidence in estimating sur-

Regularized Cox model

vival.59 Utilizing a validated survival model can aid

physicians in formulating predictions and help facilitate life
expectancy discussions with patients with advanced cancer.TagedEn
Model Type

TagedPDespite a range of prognostic models available for use in

the metastatic setting, such tools may be underutilized in
clinical practice. In a survey of radiation oncologists, only
31% rated prognostic models as moderately or very impor-
tant to them when formulating patients’ life expectancy esti-
symptomatic bone metasta-

Gensheimer 51 12,588 patients who received

397 patients who were evalu-

mates.1 Potential barriers to utilization include time

metastatic cancer care at a

AUC, area under the curve, RT, radiation therapy.

ated for palliative RT for

Retrospectively collected

Retrospectively collected
sis at a single institution

requirements and interference with clinical workflow, com-

plexity of models,26,28 and concern for lack of validity of pre-
dictions for specific patients, prognostic overconfidence, and
Patient Population
TagedEnTable 4 Artificial Intelligence Models

single institution

creation of emotional distress in patients.60 Concerns about

workflow and model complexity can be amplified for high-
dimensional and AI survival models, where model input and
output may be prohibitively intricate to permit for applica-
data

data

tion by external users in the clinic. Web and mobile plat-

forms may circumvent these limitations by facilitating ease of
data entry and display of model output.12,22 Lack of validity
of model predictions for specific subpopulations such as can-
BMETS12

didates for stereotactic RT highlights the potential need for

Alcorn;
Model

population-specific survival models in select situations.17 An

additional consideration for prognostication in the context
of stereotactic RT is that unlike most applications of
TagedEnSeminars in Radiation Oncology
conventional palliative RT, use of ablative RT in the oligome-
tastatic setting may lengthen survival time. Thus, if prognos-
tic models used in this context sway decisions toward
appropriate use of stereotactic RT—and the intervention
itself improves survival—the model’s predictive capacity
may appear to degrade over time as associations between
predictor covariates and the survival outcome weaken. This
may require repeated model refitting as well as approaches
to surveil model performance in clinical practice to ensure its
continued validity.61 Concerns about prognostic overconfi-
dence and increasing emotional distress in patients may be
mitigated by employment of shared decision making strate-
gies to ensure adequate patient education and inclusion of
patient preferences and values into the discussion of progno-
sis.60 Notably, patients do not endorse meaningful positive
or negative impact on the patient-doctor relationship on the
basis of whether prognostic information is shared.57TagedEn
TagedPMoreover, while consensus guidelines for the manage-
ment of metastatic disease uniformly highlight the role of
patient prognosis as of key importance for guiding treatment
decisions,62-66 there are no clear instructions on how to esti-
mate or incorporate patient survival into the decision making
process. For example, the American Society for Radiation
Oncology (ASTRO) recommends single-fraction palliative
RT for treatment of bone metastases in patients with limited
life expectancy.62 However, “limited” is not further quanti-
fied, nor are there recommendation on how to assess prog-
nosis. This shortcoming of available guidelines underscores
the role for decision support aids for use in the metastatic
context. Such tools are under development, including the
bone metastases ensemble trees for survival decision support
platform (BMETS-DSP) for facilitating selection of prognosis-
and guidelines-appropriate treatment for patients with symp-
tomatic bone metastases.67 This free-access, web-based deci-
sion support platform was developed to collect patient-
specific data, display a predicted survival curve based on the
BMETS model, and provide individualized evidence-based
recommendations regarding RT, open surgery, systemic ther-
apy, and hospice referral for use in clinical decision-making.
A pilot assessment of the BMETS-DSP with 10 radiation
oncology providers demonstrated that its use was associated
with improved prognostic accuracy, reduction in survival
overestimation, increased confidence in and likelihood of
sharing prognosis with the patient, and improved selection
of prognosis-appropriate RT regimens.68 Future studies are
necessary to assess its efficacy prior to widespread use. Other
decision support aids for metastatic disease are available for
specific subpopulations as well. For example, there are at
least 7 shared decision support tools for use in metastatic
breast cancer.69TagedEn
TagedPTo optimize shared clinical decision making, patient pref-
erence for if and how to receive prognostic information
should be considered as well. Evidence suggests significant
variability to the extent to which patients wish to receive pre-
dicted survival estimates—and patient willingness to receive
this information may also vary over the course of the disease
process.70 One survey of patients with metastatic cancer
demonstrated that ≥80% wanted to know their estimated
111
survival, and 59% wished to receive this information at the
time of diagnosis with metastatic disease.23 Yet a separate
survey of patients with metastatic colorectal cancer demon-
strated that a minority (44%) wanted prognostic information
presented at the time of treatment decision making, and
approximately half preferred to take a passive role in the
decision making process in general. Importantly, physicians
inaccurately predicted patient desire for receipt of prognostic
information as well as patient preference for decision auton-
omy 44% and 41% of the time, respectively.71 How data is
presented to patients is also meaningful and may shape the
nature of prognostic discussions. Surveyed patients with
metastatic cancer reported preferring discussion of survival
estimates that involved words and numbers over figures
such as charts and graphs.23 Focus groups of patients with
cancer in 1 study revealed that most consider prognostic
information to be limited to estimates of short survival time,
without having an understanding of how survival estimates
are determined and how to best use this information when
enacting lifestyle changes and making treatment decisions.
As such, patients in the study found this information to be
distressing.72 In total, these data suggest the need for an indi-
vidualized approach in which providers elicit patient desire
for if and how to discuss prognosis, including the preferred
means for presenting data and explanation for how such esti-
mates are rendered. Ideally, providers should then assess the
extent to which the patient wishes to participate in the deci-
sion process and guide patients in selecting treatment
options accordingly.TagedEn
TagedPAccurate prediction of life expectancy in patients with
metastatic cancer may lead to more appropriate selection of
treatment, such prognosis-matched palliative RT regimens.
While there are a host of available prognostic models for use
in patients with metastatic disease, appropriate selection and
application of these models should include consideration of
limitations of the underlying statistical methodology, features
of the model’s source population, and model output relative
to the clinical scenario. Development of future statistical
models should incorporate elements aimed at minimizing
barriers to clinical use of such tools given underutilization of
existing models. Patient preferences for if and how to receive
prognostic information should be integrated into the shared
decision making process for palliative RT.TagedEn
TagedH1ReferencesTagedEn
TagedP 1. Tseng YD, Krishnan MS, Sullivan AJ, et al: How radiation oncologists
evaluate and incorporate life expectancy estimates into the treatment of
palliative cancer patients: A survey-based study. Int J Radiat Oncol Biol
Phys 87:471-478, 2013. https://doi.org/10.1016/j.ijrobp.2013.06.2046TagedEn
TagedP 2. Krishnan M, Temel JS, Wright AA, et al: Predicting life expectancy in
patients with advanced incurable cancer: A review. J Support Oncol
11:68-74, 2013. https://doi.org/10.12788/j.suponc.0004TagedEn
TagedP 3. Benson KRK, Aggarwal S, Carter JN, et al: Predicting Survival for
Patients With Metastatic Disease. Int J Radiat Oncol Biol Phys 106:52-
60, 2020. https://doi.org/10.1016/j.ijrobp.2019.10.032TagedEn
TagedP 4. Hartsell WF, Desilvio M, Bruner DW, et al: Can physicians accurately
predict survival time in patients with metastatic cancer? Analysis of
RTOG 97-14. J Palliat Med 11:723-728, 2008. https://doi.org/10.1089/
jpm.2007.0259TagedEn
TagedEn112
TagedP 5. Chow E, Harth T, Hruby G, et al: How accurate are physicians’ clinical
predictions of survival and the available prognostic tools in estimating
survival times in terminally ill cancer patients? A systematic review.
Clin Oncol (R Coll Radiol) 13:209-218, 2001. https://doi.org/10.1053/
clon.2001.9256TagedEn
TagedP 6. Sborov K, Giaretta S, Koong A, et al: Impact of accuracy of survival pre-
dictions on quality of end-of-life care among patients with metastatic
cancer who receive radiation therapy. J Oncol Pract 15:e262-e270,
2019. https://doi.org/10.1200/JOP.18.00516TagedEn
TagedP 7. Guadagnolo BA, Liao KP, Elting L, et al: Use of radiation therapy in the
last 30 days of life among a large population-based cohort of elderly
patients in the United States. J Clin Oncol 31:80-87, 2013. https://doi.
org/10.1200/JCO.2012.45.0585TagedEn
TagedP 8. Stelzer KJ: Epidemiology and prognosis of brain metastases. Surg Neu-
rol Int 4(Suppl 4):S192-S202, 2013. https://doi.org/10.4103/2152-
7806.111296TagedEn
TagedP 9. Spooner A, Chen E, Sowmya A, et al: A comparison of machine learning
methods for survival analysis of high-dimensional clinical data for
dementia prediction. Sci Rep 10:20410, 2020. https://doi.org/10.1038/
s41598-020-77220-wTagedEn
TagedP10. Cox DR: Regression models and life-tables. J Royal Statis Soc Ser B
(Methodological) 34:187-202, 1972TagedEn
TagedP11. Biau G: Analysis of a random forests model. J Mach Learn Res 13:1063-
1095, 2012TagedEn
TagedP12. Alcorn SR, Fiksel J, Wright JL, et al: Developing an improved statistical
approach for survival estimation in bone metastases management: The
bone metastases ensemble trees for survival (BMETS) model. Int J
Radiat Oncol Biol Phys 108:554-563, 2020. https://doi.org/10.1016/j.
ijrobp.2020.05.023TagedEn
TagedP13. Dankowski T, Ziegler A: Calibrating random forests for probability esti-
mation. Stat Med 35:3949-3960, 2016. https://doi.org/10.1002/
sim.6959TagedEn
TagedP14. Steyerberg EW, Vickers AJ, Cook NR, et al: Assessing the performance
of prediction models: A framework for traditional and novel measures.
Epidemiology 21:128-138, 2010. https://doi.org/10.1097/EDE.0b013
e3181c30fb2TagedEn
TagedP15. Harrell FE, Lee KL, Mark DB: Multivariable prognostic models: issues in
developing models, evaluating assumptions and adequacy, and measuring
and reducing errors. Stat Med 15:361-387, 1996. https://doi.org/10.1002/
(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4TagedEn
TagedP16. Hilden J, Habbema JD, Bjerregaard B: The measurement of performance
in probabilistic diagnosis. II. Trustworthiness of the exact values of the
diagnostic probabilities. Methods Inf Med 17:227-237, 1978TagedEn
TagedP17. LaVigne A, Fiksel J, Wright J, et al. Evaluating and Optimizing Prognos-
tic Modeling for Patients Undergoing Stereotactic Body Radiotherapy
for Bone Metastases. Presented at ASTRO 2022.TagedEn
TagedP18. Patchell RA, Tibbs PA, Regine WF, et al: Direct decompressive surgical
resection in the treatment of spinal cord compression caused by meta-
static cancer: A randomised trial. Lancet 366:643-648, 2005. https://
doi.org/10.1016/S0140-6736(05)66954-1TagedEn
TagedP19. Howell DD, Lutz S: Hospice referral: An important responsibility of the
oncologist. J Oncol Pract 4:303-304, 2008. https://doi.org/10.1200/
JOP.0841501TagedEn
TagedP20. Rades D, Panzner A, Rudat V, et al: Dose escalation of radiotherapy for
metastatic spinal cord compression (MSCC) in patients with relatively
favorable survival prognosis. Strahlenther Onkol 187:729-735, 2011.
https://doi.org/10.1007/s00066-011-2266-yTagedEn
TagedP21. Howell DD, James JL, Hartsell WF, et al: Single-fraction radiotherapy
versus multifraction radiotherapy for palliation of painful vertebral
bone metastases-equivalent efficacy, less toxicity, more convenient: a
subset analysis of Radiation Therapy Oncology Group trial 97-14. Can-
cer 119:888-896, 2013. https://doi.org/10.1002/cncr.27616TagedEn
TagedP22. Zaorsky NG, Liang M, Patel R, et al: Survival after palliative radiation
therapy for cancer: The METSSS model. Radiother Oncol 158:104-111,
2021. https://doi.org/10.1016/j.radonc.2021.02.011TagedEn
TagedP23. Hagerty RG, Butow PN, Ellis PA, et al: Cancer patient preferences for
communication of prognosis in the metastatic setting. J Clin Oncol
22:1721-1730, 2004. https://doi.org/10.1200/JCO.2004.04.095TagedEn
S. Sun et al.
TagedP24. Chow E, Fung K, Panzarella T, et al: A predictive model for survival in
metastatic cancer patients attending an outpatient palliative radiother-
apy clinic. Int J Radiat Oncol Biol Phys 53:1291-1302, 2002. https://
doi.org/10.1016/s0360-3016(02)02832-8TagedEn
TagedP25. Krishnan MS, Epstein-Peterson Z, Chen YH, et al: Predicting life expec-
tancy in patients with metastatic cancer receiving palliative radiother-
apy: The TEACHH model. Cancer 120:134-141, 2014. https://doi.org/
10.1002/cncr.28408TagedEn
TagedP26. Westhoff PG, de Graeff A, Monninkhof EM, et al: An easy tool to predict
survival in patients receiving radiation therapy for painful bone metasta-
ses. Int J Radiat Oncol Biol Phys 90:739-747, 2014. https://doi.org/
10.1016/j.ijrobp.2014.07.051TagedEn
TagedP27. Katagiri H, Okada R, Takagi T, et al: New prognostic factors and scoring
system for patients with skeletal metastasis. Cancer Med 3:1359-1367,
2014. https://doi.org/10.1002/cam4.292TagedEn
TagedP28. Chow E, Abdolell M, Panzarella T, et al: Predictive model for survival in
patients with advanced cancer. J Clin Oncol 26:5863-5869, 2008.
https://doi.org/10.1200/JCO.2008.17.1363TagedEn
TagedP29. Chow E, Abdolell M, Panzarella T, et al: Recursive partitioning analysis
of prognostic factors for survival in patients with advanced cancer. Int J
Radiat Oncol Biol Phys 73:1169-1176, 2009. https://doi.org/10.1016/j.
ijrobp.2008.05.067TagedEn
TagedP30. Dosani M, Tyldesley S, Bakos B, et al: The TEACHH model to predict
life expectancy in patients presenting for palliative spine radiotherapy:
External validation and comparison with alternate models. Support
Care Cancer 26:2217-2227, 2018. https://doi.org/10.1007/s00520-
018-4064-xTagedEn
TagedP31. Kubota H, Soejima T, Sulaiman NS, et al: Predicting the survival of
patients with bone metastases treated with radiation therapy: A valida-
tion study of the Katagiri scoring system. Radiat Oncol 14:13, 2019.
https://doi.org/10.1186/s13014-019-1218-zTagedEn
TagedP32. Christ SM, Schettle M, Willmann J, et al: Validation and extension of the
METSSS score in a metastatic cancer patient cohort after palliative
radiotherapy within the last phase of life. Clin Transl Radiat Oncol
34:107-111, 2022. https://doi.org/10.1016/j.ctro.2022.04.005TagedEn
TagedP33. Rades D, Dunst J, Schild SE: The first score predicting overall survival in
patients with metastatic spinal cord compression. Cancer 112:157-161,
2008. https://doi.org/10.1002/cncr.23150TagedEn
TagedP34. Rades D, Douglas S, Veninga T, et al: Validation and simplification of a
score predicting survival in patients irradiated for metastatic spinal cord
compression. Cancer 116:3670-3673, 2010. https://doi.org/10.1002/
cncr.25223TagedEn
TagedP35. Rades D, Douglas S, Veninga T, et al. Erratum to: Validation and simpli-
fication of a score predicting survival in patients irradiated for metastatic
spinal cord compression. Cancer 128:633-634, 2022. https://doi.org/
10.1002/cncr.33941TagedEn
TagedP36. Rades D, Hueppe M, Schild SE: A score to identify patients with meta-
static spinal cord compression who may be candidates for best support-
ive care. Cancer 119:897-903, 2013. https://doi.org/10.1002/
cncr.27849TagedEn
TagedP37. van der Linden YM, Dijkstra SP, Vonk EJ, et al: Prediction of survival in
patients with metastases in the spinal column: Results based on a ran-
domized trial of radiotherapy. Cancer 103:320-328, 2005. https://doi.
org/10.1002/cncr.20756TagedEn
TagedP38. Lee SF, Luk H, Wong A, et al: Prediction model for short-term mortality
after palliative radiotherapy for patients having advanced cancer: a
cohort study from routine electronic medical data. Sci Rep 10:5779,
2020. https://doi.org/10.1038/s41598-020-62826-xTagedEn
TagedP39. V
azquez M, Altabas M, Moreno DC, et al: 30-day mortality following
palliative radiotherapy. Front Oncol 11, 2021:668481. https://doi.org/
10.3389/fonc.2021.668481TagedEn
TagedP40. Wu SY, Yee E, Vasudevan HN, et al: Risk stratification for imminent risk
of death at the time of palliative radiotherapy consultation. JAMA Netw
Open 4, 2021:e2115641. https://doi.org/10.1001/jamanetworkopen.
2021.15641TagedEn
TagedP41. van der Velden JM, Peters M, Verlaan JJ, et al: Development and internal
validation of a clinical risk score to predict pain response after palliative
radiation therapy in patients with bone metastases. Int J Radiat Oncol
TagedEnSeminars in Radiation Oncology
Biol Phys 99:859-866, 2017. https://doi.org/10.1016/j.ijrobp.2017.
07.029TagedEn
TagedP42. van der Velden JM, Peters M, Verlaan J-J, et al: Erratum to: Develop-
ment and internal validation of a clinical risk score to predict pain
response after palliative radiation therapy in patients with bone metasta-
ses. Int J Radiat Oncol Biol Phys 104(5):1186, 2019. https://doi.org/
10.1016/j.ijrobp.2019.05.016TagedEn
TagedP43. Westhoff PG, de Graeff A, Monninkhof EM, et al: Quality of life in rela-
tion to pain response to radiation therapy for painful bone metastases.
Int J Radiat Oncol Biol Phys 93:694-701, 2015. https://doi.org/
10.1016/j.ijrobp.2015.06.024TagedEn
TagedP44. Tree AC, Khoo VS, Eeles RA, et al: Stereotactic body radiotherapy for
oligometastases. Lancet Oncol 14:e28-e37, 2013. https://doi.org/
10.1016/S1470-2045(12)70510-7TagedEn
TagedP45. Lewis SL, Porceddu S, Nakamura N, et al: Definitive stereotactic body
radiotherapy (SBRT) for extracranial oligometastases: An international
survey of >1000 radiation oncologists. Am J Clin Oncol 40:418-422,
2017. https://doi.org/10.1097/COC.0000000000000169TagedEn
TagedP46. Tang C, Hess K, Bishop AJ, et al: Creation of a prognostic index for
spine metastasis to stratify survival in patients treated with spinal stereo-
tactic radiosurgery: Secondary analysis of mature prospective trials. Int J
Radiat Oncol Biol Phys 93:118-125, 2015. https://doi.org/10.1016/j.
ijrobp.2015.04.050TagedEn
TagedP47. Jensen G, Tang C, Hess KR, et al: Internal validation of the prognostic
index for spine metastasis (PRISM) for stratifying survival in patients
treated with spinal stereotactic radiosurgery. J Radiosurg SBRT 5:25-34,
2017TagedEn
TagedP48. Tanadini-Lang S, Rieber J, Filippi AR, et al: Nomogram based overall
survival prediction in stereotactic body radiotherapy for oligo-metastatic
lung disease. Radiother Oncol 123(2):182-188, 2017. https://doi.org/
10.1016/j.radonc.2017.01.003TagedEn
TagedP49. Elledge CR, LaVigne AW, Fiksel J, et al: External validation of the bone
metastases ensemble trees for survival (BMETS) machine learning model to
predict survival in patients with symptomatic bone metastases. JCO Clin
Cancer Inform 5:304-314, 2021. https://doi.org/10.1200/CCI.20.00128TagedEn

TagedP50. Nieder C, Mannsaker B, Yobuta R: Independent validation of a compre-
hensive machine learning approach predicting survival after radiother-
apy for bone metastases. Anticancer Res 41:1471-1474, 2021. https://
doi.org/10.21873/anticanres.14905TagedEn
TagedP51. Gensheimer MF, Henry AS, Wood DJ, et al: Automated survival predic-
tion in metastatic cancer patients using high-dimensional electronic
medical record data. J Natl Cancer Inst 111:568-574, 2019. https://doi.
org/10.1093/jnci/djy178TagedEn
TagedP52. Weeks JC, Cook EF, O’Day SJ, et al: Relationship between cancer patients’
predictions of prognosis and their treatment preferences. JAMA 279:1709-
1714, 1998. https://doi.org/10.1001/jama.279.21.1709TagedEn
TagedP53. Wright AA, Zhang B, Ray A, et al: Associations between end-of-life dis-
cussions, patient mental health, medical care near death, and caregiver
bereavement adjustment. JAMA 300:1665-1673, 2008. https://doi.org/
10.1001/jama.300.14.1665TagedEn
TagedP54. Teno JM, Clarridge BR, Casey V, et al: Family perspectives on end-of-life
care at the last place of care. JAMA 291:88-93, 2004. https://doi.org/
10.1001/jama.291.1.88TagedEn
TagedP55. Zhang B, Wright AA, Huskamp HA, et al: Health care costs in the last
week of life: Associations with end-of-life conversations. Arch Intern Med
169:480-488, 2009. https://doi.org/10.1001/archinternmed.2008.587TagedEn
TagedP56. Epstein AS, Prigerson HG, O’Reilly EM, et al: Discussions of life expec-
tancy and changes in illness understanding in patients with advanced
cancer. J Clin Oncol 34:2398-2403, 2016. https://doi.org/10.1200/
JCO.2015.63.6696TagedEn
113
TagedP57. Enzinger AC, Zhang B, Schrag D, et al: Outcomes of prognostic
disclosure: Associations with prognostic understanding, distress,
and relationship with physician among patients with advanced can-
cer. J Clin Oncol 33:3809-3816, 2015. https://doi.org/10.1200/
JCO.2015.61.9239TagedEn
TagedP58. Daugherty CK, Hlubocky FJ: What are terminally ill cancer patients told
about their expected deaths? A study of cancer physicians’ self-reports
of prognosis disclosure. J Clin Oncol 26:5988-5993, 2008. https://doi.
org/10.1200/JCO.2008.17.2221TagedEn
TagedP59. Mack JW, Smith TJ: Reasons why physicians do not have discussions
about poor prognosis, why it matters, and what can be improved. J Clin
Oncol 30:2715-2717, 2012. https://doi.org/10.1200/JCO.2012.42.
4564TagedEn
TagedP60. Hallen SA, Hootsmans NA, Blaisdell L, et al: Physicians’ perceptions of
the value of prognostic models: The benefits and risks of prognostic
confidence. Health Expect 18:2266-2277, 2015. https://doi.org/
10.1111/hex.12196TagedEn
TagedP61. Lenert MC, Matheny ME, Walsh CG: Prognostic models will be victims
of their own success, unless. . .. J Am Med Inform Assoc 26:1645-1650,
2019. https://doi.org/10.1093/jamia/ocz145TagedEn
TagedP62. Lutz S, Balboni T, Jones J, et al: Palliative radiation therapy for bone
metastases: Update of an ASTRO evidence-based guideline. Pract Radiat
Oncol 7:4-12, 2017. https://doi.org/10.1016/j.prro.2016.08.001TagedEn
TagedP63. Kim EY, Chapman TR, Ryu S, et al: ACR Appropriateness Criteria(Ò )
non-spine bone metastases. J Palliat Med 18:11-17, 2015. https://doi.
org/10.1089/jpm.2014.9395TagedEn
TagedP64. Lo SS, Lutz ST, Chang EL, et al: ACR Appropriateness Criteria Ò spinal
bone metastases. J Palliat Med 16:9-19, 2013. https://doi.org/10.1089/
jpm.2012.0376TagedEn
TagedP65. Lo SS, Ryu S, Chang EL, et al: ACR Appropriateness CriteriaÒ metastatic
epidural spinal cord compression and recurrent spinal metastasis. J Pall-
iat Med 18:573-584, 2015. https://doi.org/10.1089/jpm.2015.28999.
smlTagedEn
TagedP66. NCCN Guidelines. Available at: https://www.nccn.org/guidelines/cate-
gory_1. Accessed February 5, 2023.TagedEn
TagedP67. Alcorn SR, Elledge CR, LaVigne AW, et al: Improving providers’ sur-
vival estimates and selection of prognosis- and guidelines-appropriate
treatment for patients with symptomatic bone metastases: Development
of the bone metastases ensemble trees for survival decision support plat-
form. J Eval Clin Pract 28:581-598, 2022. https://doi.org/10.1111/
jep.13652TagedEn
TagedP68. SR Alcorn, J Fiksel, C Hu, et al., Pilot assessment of the BMET decision
support platform: A tool to improve provider survival estimates and
selection of prognosis-appropriate treatment for patients with symp-
tomatic bone metastases, Int J Radiat Oncol Biol Phys, 105(1S):S47,
2019.TagedEn
TagedP69. Spronk I, Burgers JS, Schellevis FG, et al: The availability and effective-
ness of tools supporting shared decision making in metastatic breast
cancer care: A review. BMC Palliat Care 17:74, 2018. https://doi.org/
10.1186/s12904-018-0330-4TagedEn
TagedP70. Russell BJ, Ward AM: Deciding what information is necessary: Do
patients with advanced cancer want to know all the details? Cancer
Manag Res 3:191-199, 2011. https://doi.org/10.2147/CMR.S12998TagedEn
TagedP71. Elkin EB, Kim SH, Casper ES, et al: Desire for information and involve-
ment in treatment decisions: Elderly cancer patients’ preferences and
their physicians’ perceptions. J Clin Oncol 25:5275-5280, 2007.
https://doi.org/10.1200/JCO.2007.11.1922TagedEn
TagedP72. Cartwright LA, Dumenci L, Siminoff LA, et al: Cancer patients’ under-
standing of prognostic information. J Cancer Educ 29:311-317, 2014.
https://doi.org/10.1007/s13187-013-0603-9TagedEn